No, you pretty much nailed it. My moral compass is not based on any kind of “belief,” but entirely upon my owm emotional responses to my own actions. I nurture my children because it makes me feel good to nurture my children, not because I think there’s any kind of rule about it or because “I believe it is right.” I avoid hurting people because it makes me feel bad to hurt people, not because I think it’s violates any sort of external rule.
This is the part where somebody brings up serial killers or Nazis and asks if they’re just following their own moral compass and what makes mine more “valid” than theirs. The answer to the first question is yes, the answer to the second is that the word “validity” has no application to morality any more than it has any application to taste in beer.
The next question then is what gives anybody the right or authority to place any behavioral parameters on others? (If you think rape is wrong and I think it’s right, who are you to tell me I can’t do it?) The answer is that the “authority” (for lack of a better word) comes from your own moral sense and needs to come from nowhere else. If you feel morally obligated to protect others from rape, then that’s what you do. It doesn’t have to be justified. You’re just following your own code. If that conflicts with someone else, then there’s going to be a power struggle and whoever wins, wins.
Fortunately, this doesn’t lead to the kind of anarchy and chaos that it sounds like it might because, as it turns out, a lot of our emotional responses to such things are hardwired into our biology and basically shared. Most people have an empathic response which causes them to feel distress and anxiety at the suffering of others. That means it’s pretty easy for the vast majority to agree that we can pool our collective power to try to prevent a microspcopic minority from engaging in behavior which destabalizes our communities or causes us severe emotional distress.