I wouldn’t exactly say rude, but it is alienating both to the person who doesn’t order and to the one(s) who do.
The one time it happened to me was when I chaired a committee and we met over dinner (I had been a member of the committee for several years and this was SOP–it made membership more bearable). It turned out that one of the new members was strictly kosher. He eventually ordered ice cream when the rest of us got desert. I didn’t think it rude but it left me distinctly uncomfortable. The next two years, we didn’t have dinner meetings and then I was gone so I have no idea if they resumed when he left–probably not.
For me it’s always the opposite. When I go to a cafe with a bunch of friends, I know we’re all supposed to just order a drink. But I’m the only one who always orders food along with it too, and I wonder how that makes me look. Wonder if it’s rude to be the only one who orders food 
More like horrifying. (as I’m sure you know) A lot of people around here are just a generation or two away from times of, if not starvation, really really bad hunger. If you turn your nose up at food that such folks are now offering to share with you, it’s like you’re throwing the whole table in their faces or something. Only a bit of hyperbole: even nowadays the common greeting in Chinese & Taiwanese is not “hello,” but “have you eaten yet?”
Between this and being expected to take your shoes off when you enter someone’s house, I’ve come to believe that your average free spirited American would have their heads explode in short order in this society.
I do it all the time. I react poorly to onions - contained in nearly every restaurant food item - and find most restaurant food ridiculously unhealthy, greasy, and lacking in vegetables. But I always forewarn people when the plans involve a restaurant that I’ll probably just order drinks.
Of course, as has been pointed out, I would never just not order at a fine dining restaurant, or at a dinner when the whole point is to be eating. For dinner at the French Laundry and Alinea and Le Chateaubriand and many others, I ate without worrying too much about the onions or the amount of butter, and the dinners were amazing. But I’m not going to eat for the sake of eating when it’s just a pub with friends and I don’t really enjoy any of the food.
It’s rude to fill yourself up to the point that you can’t eat a single bite when you’re planning to go out to eat with people, unless you’ve told them in advance that that’s what you’re going to do. You should leave enough room to be able to eat something.
My take: not rude in the sense that others would be entitled to take offense, but rude in the sense that you’d best avoid it, to minimize awkwardness.
I don’t recollect the situation of the OP (being full at the start of a social meal) ever having occurred to me - social eating occasions don’t usually occur suddenly for me; I can schedule time/portions of my other meals so as to be able to do justice to the occasion. (e.g. yesterday I met with friends where one had baked a cake for tea and another had prepared homemade pizza for dinner - you bet I took care not to arrive already sated).
If I were ever caught out in such a situation I’d order an inexpensive appetizer and a glass of water, and make them last. When people had prepared food themselves, I’d eat a polite portion or perish in the attempt.
What if you’re a Kreetassan?
As Lora Brody wrote in her book on entertaining: For any number of reasons, a guest may refuse your offer of a drink. The reason doesn’t have to be explained to anyone, including you.
I think that applies to food too. I know a woman who eats the same things every day: bagel and coffee for breakfast, a pint of frozen dessert for lunch, and a big vegetable salad for supper. If she can’t find what she wants when we eat out, she doesn’t eat.
To me, “eating is a social activity” and “the whole point is to be eating” are mutually exclusive propositions. Socialization is supported by eating and drinking together, but it mostly takes place in the interstices, not when everybody’s mouths are full. It seems that the “rude” element of not-eating or not-drinking in such a context comes from the unequal position of the participants; the non-eater has no parallel activity to modulate their conversation and open spaces for their companions.
Seems to me a bit like going to someone’s Super Bowl party and bringing along a book to read.
Not an ex-server here, but if I’m not eating I’ll at least order a beverage and leave a tip. For much the same reason.
I try to be good to the wait staff. After all, they’re people, too, and have to put up with the general public.
But, between trying to stay at a normal weight and my dietary restrictions for medical reasons I simply won’t put up with someone trying to force food on me in the name of “politeness”. “Polite” is not forcing people into weight problems, risking their health, or imposing your culture on them.*
- Yes, I realize there probably are a few exceptions to the latter - but they’re relatively rare and would be something like a wedding or traditional/custom-bound event that you’d probably know about before showing up.
Well, given THAT social context I’d of course take my shoes off and eat at least a token amount of something. Fortunately, I can eat most Asian foods without requiring a trip to the emergency room.
(Heck, I take my shoes off as soon as I get into my own house, no biggie)
It depends on the situation, and SanVito’s comment pretty much expresses how I feel about it.
Most eating out seems like it’s a social situation, though, and I’d probably expect that, even if the person weren’t eating, at least they’d order a cocktail or soda or something other than water while I ate.
Part of that is just baggage, though. I do NOT like to eat in front of people if no one else is eating. It makes me extremely, extremely uncomfortable.
The only time I think it’s rude is when I’m with one or two friends and ask them, say while we’re in the car driving home, “want to stop at Armando’s/IHOP/Applebee’s/wherever?” and they say “sure!” and only when we arrive do I discover that they don’t plan on actually eating anything.
In fact, it pisses me the fuck off. Yes, I get it, maybe you don’t want to spend a lot of money eating. But you could have just fucking said “no”. Thankfully they haven’t done it after I’ve made my feelings known to them.
I’ve done that before…but it was one of those incidental “hey let’s go to Perkins!” on-the-fly type activities. It was after prom, and they’d served us food there…food my stomach wasn’t really all that sure about. So when I went out afterwards, I ordered a cup of tea, and nibbled on a fry or two when a friend offered me some. No one batted an eye.
So put me in the “not-rude” camp, though like others have said, it’s a bit weird when the eating is the point of the activity.