What happens to a stateless person in the US?

Me too, and my wife. It was once so, but there was some SCOTUS decision banning the US from taking citizenship away from people who naturalized elsewhere. Not sure when it changed, but sometime between 1987 and 2007. In 1986, my two older children naturalized here because Canada allowed 18 year olds to naturalize while the US didn’t take away citizenship of people under 21. 2007 was when my wife became Canadian.

If you will pardon the hijack, I’d like to tell an amusing story. Once we were crossing into the US and the border agent asked where we were born. This was in the days you didn’t need a passport between Canada and US. I pointed to the oldest and said that she was US born, then to middle one and said he was born in Switzerland and to the youngest and said that he was born in Canada. He looked at me and said, “What are you, some kind of professor?” “Guilty as charged.”

I had to look that up. I’m afraid I fell into the roestigraben!

The scenario described in the OP applies to a handful of former Nazis & collaborators. They went underground at the end of WWII, got naturalized in the US, and were decades later discovered and stripped of their US citizenship. However, they couldn’t be deported anywhere else either, because they only had US citizenship at that point. At least a few of them died in the US in the end.

Ah the Roestigraben. That was the informal name of the Sarine River that flowed through Fribourg where I spent a year. It was said to divide Fribourg into the French and German parts of the Canton. It didn’t really. The town of Marly where I lived was largely French speaking although on the German side of the river.