I suppose I do ask others this indirectly, if it arises naturally in the conversation … it is part of getting to know people. But at least in the UK, to come right out and ask this of someone you’ve just met would be considered not rude exactly, but a little bit uncouth. Maybe like asking someone how old they are: it’s hardly secret information, but does trigger a bit of a ‘why do you care?’ response.
Now imagine upon finding out that you are 40 they start droning on about the other people they know in their 40s, or bring out the crass stereotypes about 40 year-olds …
My objection (and, I think, the OPs), is to people who either interrupt you or say it right upon meeting you for the first time, because you look or talk funny. That is more annoying, it carries a connotation of ‘hey you! something’s odd about you and I’m gonna figure out what that is!’. I don’t care if someone ‘is curious about accents’ or whatever, this is rude.
I have noticed that (white) Americans use ‘where are you from’ as a standard greeting/piece of smalltalk more than British people, and there’s nothing wrong with it. But when you are being asked in a slightly more inquisitive way because you don’t fit in in some way, and they won’t let you dodge the question, with some power imbalances (it is their country, after all, so you don’t want to be too rude), it gets tiresome.
I know that some people like to be the centre of attention, and to talk about themselves to new people. I don’t, really, and the problem with a lot of people who ask the question is that they won’t let it drop, and they won’t let you get away with an evasive answer… they will keep pushing until you tell them. (The other option is to put your foot down, but then you look like an oversensitive jerk.) So it’s fine for the extroverts, but for the rest of us it really is like being put unwelcomely on the spot.
But as I say, just one of life’s minor annoyances.
pdts