I agree that context matters, and needs to be part of the discussion of any particular person’s experience.
When we’re talking about a “Muslim ban,” it’s fair to call that racist. No one is looking to ban Muslims born in England who have blonde hair. They’re also not keen on allowing entry of Iranians who are also Christian (yes, there are some). So, in that context, “Muslim” is a racial label that one can’t escape by the simple action of not being religious. And it seems right now to be filling - albeit ill-fittingly - a racial category in the US that doesn’t have another name - those people from the Middle East who aren’t culturally/ethnically/racially “White,” but we don’t have a non-religious race name for them yet.
If a stand up comedian is making jokes about his childhood days at madrassa or the way his father interrupted his soccer game to pray, then maybe I’d refer to that person as Muslim, even if now atheist, because his Muslim heritage is relevant to his current work.
If we’re asking specifically, “what’s your religion?” and a person says, “I don’t have one, I’m atheist,” then I wouldn’t call him a Muslim, because this context is about his personal religion.
So the very same person might be Muslim, Muslim, and Not a Muslim, depending on the conversation we’re having.
As for ex Catholics, Lutherans, et al, what I generally hear from them is, “I was raised Lutheran,” or “I’m a former Lutheran,” or “lapsed Catholic,” or even, “recovering Catholic.” Modifiers that let you know they were once of that religion, but are not anymore.
I do agree that Jewish people are different than other religions, because their ethnicity and their religion share the same name, more often than not. Yes, there are Ethiopian Jews and Black Jews - but in my experience, the modifiers are always used when referring to them.
I’ve never, ever heard a person refer to themselves as a “former Jew.” They may say, “I’m Jewish, but not religious,” or “I’m culturally Jewish,” but it seems once a Jew, always a Jew, even if an atheist.