Max, if everyone were like you during the time of slavery, few or zero slaves would have successfully escaped, because no one would have helped them and many would have turned them in. Do you believe that would have been a better and more moral outcome than what actually happened in history (i.e. Many slaves escaped due to assistance from abolitionists and others)?
If not, then you already recognize that your moral system is unacceptable to some degree.