i mean non-anglicized foreign words; e.g. gestalt to gestalten or gestalts, antenna to antennae or antennas, etc.
How is antenna a non-anglicized foreign word?
I don’t know the general rule, but I just look it up in the dictionary. Seems simple enough. Gestalts and gestalten both look to be correct.
This is just my personal opinion and there are certain to be disagreements: Yes, pluralize them following the rules of English, by adding s or es, as appropriate. In speech this can come out as z, s, or ez, by the way. I have a number of reasons for this, but the most important is the habit of foreign plurals becoming used as singular: data, media, I have even heard phenomena used as a singular (or phenomenas used for the plural). There are others, but they don’t come to mind right now. The usage of foreign plurals is mainly confined to words from Latin or Greek anyway and usually is a display of (false) erudition.
That said, it still feels odd to say phenomenons and calculuses, but I do it anyway. But datums is beyond me and mediums has acquired a special meaning.
Funguses and genuses I can live with. Bacteriums? Algas? Radiuses? Opuses? 
There are some well-known words, many of which have already been listed in this thread, that should probably be pluralized in accord with the language they were borrowed from, because both they and their plurals have been adopted into English as irregular nouns. A decent amount of reading and a few slip-ups around erudite assholes will teach you all of them. For any other random foreign word, though, you should use the standard English pluralization rules outlined by Hari Seldon (the -es variant is used with words that end in the sounds /s/ or /z/; all other words use -s) (I think).
I pluralized equinox as equinoxen in a recent thread. Some people didn’t get it.
There’s an upcoming Staff Report on this general topic. The short answer is that there is no hard and fast rule. The best thing to do is to look up the word in a decent dictionary that gives the plural form if there can be any doubt (not all dictionaries do this, unfortunately). The American Heritage Dictionary is pretty good for this, but not perfect. If there are two choices given for the plural, the first choice is generally preferred unless otherwise specified.
Why do you say it’s not perfect? Does it get some things wrong?
If the word comes from a language that does not pluralize nouns, such as Japanese, I think it’s more natural to not pluralize when using the word in English.
One samurai, two samurai, three samurai: The Last Three Samurai.
If the foreign word is an un-countable noun, it doesn’t make sense to pluralize it in English, because un-countable nouns are treated as singular nouns in English.
Some sushi, a lot of tempura, and a little bit of tofu.
It does make sense in at least some instances. Virus is one example that springs to mind. It would be very hard to talk about different strains of a virus and then go on to discuss different species being infected with different virus. It sounds like you are speaking of only one ‘species’ of virus when in fact you mean that different species are infected with different viruses.
In reality I think it depends how completely the word has been grafted onto English. While Samurai in many ways is only a loan word and quite rightly imperfectly pluralised other words such as virus and Toyota have become true parts of the English language. I’m sure we’d all find it odd to see a competition offering the chance to win “Two new Toyota”.
Maybe not a GQ answer but: as Hari Seldon says, a lot of people solve the problem with some words by never using the singular form. Data became singular worldwide (outside the scientific community) some time ago, but I also hear “a bacteria” and “a criteria” a lot. In a recent documentary about the legal system, several presumably highly-educated lawyers and a judge were involved in assessing criteria for a particular form of legal treatment. Not one of them used the singular of criterion. At least the narrator did.
This use of the plural for the singular really hit me when I moved to the US, so I think it is more prevalent here than in my native England.
As for the OP, I go with antennae etc.