Yeah. If we can’t characterize a group that has exceptions to the characterization, then we pretty much can’t discuss groups at all. That’s a standard rhetorical trick the Right likes to use to shut down discussion of it, I see it all the time.
Then there’s the issue that we can’t read minds, only judge by actions. Claiming purity of motive is meaningless, both because the rest of us can’t tell and because results are what matters.
And on top of that since personal attacks will get you into trouble it’s rather disingenuous for someone to defend a group by claiming themself as a counterexample, since nobody is allowed to contradict them (outside of the Pit at least). If it was a third party at least we could have an actual discussion about it.