No. The point is not how bad the label is, but what the threshold is that one might pass in order to be accurately characterized as such. So, for instance, someone arguing for allowing more religious expression in the public square, may or may not be a Christian. Someone arguing against abortion, may or may not be a Christian. Someone discussing whether all races have the same cognitive abilities may or not be a racist.
Not only that, given the seriousness of the accusation of “racist”, don’t you think the bar for using it should be pretty fucking high? Or do you think using it more cavalierly is just fine?
But shouldn’t we determine first that it actual hate speech?