This is actually true. And it is why it is stupid when someone sees an article about a particular group of people engaging in violence or other anti-social type behavior, and then broad brushes the rest of the group with that behavior.
That’s why when I judge a group, I look to its leaders and its stated principles, not isolated cases of misguided followers.
If a person is reprehensible, a racist, a bigot, or whatever, and people are following that person, with the full knowledge of that person’s views, then the group can fairly be described as racist. If a group’s stated purpose is to ensure the supremacy of the white race, then you can fairly describe the group a white supremacist. If the leaders of a group call for violence, or talk about it in fond terms after it happens, then you can fairly describe the group as violent.
If you take the actions of a follower of the group, and the group disavows those actions, but you choose to brush the group with those action anyway, that is when you get to broad brushing fallacies that you are very fond of spouting.
You can say about antifa, that they are not always violent, and in fact, are only violent in the cases where you heard about the violence. So if you describe antifa as a violent group, you are wrong, there are many times and places that they do not engage in violence.
Nazis, on the other hand, at the core of their ideology, is the eradication of anyone that they feel is of an inferior race. That is the case for the leaders, the followers, and everyone in between. You can not say about a nazi group, ever, that they are not for genocide, that’s their calling card.