Well, yes and no. It gets messy, because being Amish is both about religion and ethnicity. I’m not Amish, nor have I even met any Amish people, but I am/was Mennonite. I’m not from one of the technology-eschewing branches; no horse-drawn carriages or black suits without buttons for us, but ethnically distinct all the same.
I am an atheist. I am no longer a Christian of any description, and therefore certainly not a Mennonite in the religious sense. Oh, I still am very sympathetic to many of the basic moral principles they hold important - peacefulness, simple living, etc., but that’s certainly not enough to qualify. However, I’m certainly still Mennonite in some sense. I mean, how would you describe my ethnicity? Canadian? Sure, I’m a 4th generation Canadian. But that seems to be leaving something out, since every last one of my ancestors was Mennonite (they/we are not a particularly exogamous group). But what, then? German? The description is used often enough, because the German language (in both “High” and “Low”, i.e., Plattdietsch, variants) is used by the Mennonite community. It’s not very accurate, though. My ancestors first became an identifiable group in the Low Countries in the 16th Century. So, Dutch, maybe? Not really. The Mennonites were almost entirely wiped out in the Low Countries. Persecuted by Catholics and Protestants alike, those that didn’t revert to those denominations fled to Prussia, where some noblemen around Danzig (present day Gdansk) on the Vistula river delta were more interested in swamp-draining expertise than they were in people’s views on the appropriate mode of baptism. Eventually, though, strict pacifism ran afoul of compulsory military service, and the Mennonites in Prussia either assimilated or migrated, the latter accepting an offer from Catherine the Great to colonize newly conquered lands in the Ukraine. Presumably it was at this point that we picked up our penchant for borscht and verenike (which the rest of the world calls perogies, though we fill ours with cottage cheese, not potato). So Ukrainian? Not really. The Mennonites in the Ukraine remained extremely insular (though I believe that has changed since WWII), so much so that there’s only a tiny smattering of Ukrainian names to be found in the large Mennonite community in western Canada who eventually left the Ukraine to homestead in Manitoba and Saskatchewan. For 400 years, my people have been an identifiable group isolated to a large extent from the populations surrounding them, so there simply isn’t any description for my ethnicity except Mennonite.
The story of the Amish is different, starting in south Germany and Switzerland, and doesn’t involve wandering across most of Europe, but the situation is quite similar. If a person ceases to follow the Amish religion, would they then cease to be Amish? Yes. And no. Am I still Mennonite? No. But yes.