I don’t think it is hate speech. It *would *veer that way if someone said something like “Black Republicans are all Uncle Toms”, and definitely if you tacked on “… and we should get them for it” but that isn’t how it is used. I know when I’ve used it, I’ve directed it at a specific person for specific actions. “Colin Powell is an Uncle Tom for his WMD speech to the UN” is not hate speech, it’s an opinion about Powell’s actions, and doesn’t even slightly imply taking any action towards Powell either, which is part&parcel of the hate speech qualifier to me.
And I have used, and will use, the term or a variant thereof, for people who aren’t Black. Although usually I find “House <whatever>” works better. Trying to remember when I used the term “House Gay”…