A couple points.
First, an “abolitionist” is simply someone who supports the abolition of slavery in their own country. They do not necessarily have to believe that slavery is evil. They might, for example, be a racist who wanted white workers to have the jobs which were performed by black slaves. Or they might have simply wanted all blacks expelled from the country, to cleanse it for white society, which could only have happened if their economic utility as slaves was removed. Or they may have felt that it was “evil” in a sense, but because of its effect on the slaveholder, not necessarily the slave.
All of these views were expressed at one time or another by white Americans between the War of Independence and the War Between the States. Lincoln subscribed to the first, at least for a time, though he was not originally (if we take him at his word) an abolitionist at all, but simply opposed to the expansion of slavery into new western territories.
Robert E. Lee, on the other hand, did “get it,” (at least in his own terms) as he came to view slavery as sinful (a view somewhat akin to the third outlook above), and eventually freed the slaves he had received from his wife’s family. (The Custis estate was mismanaged and in debt, making manumission difficult at first. Some former slaves chose to remain with the Lees as free servants.) In an 1856 letter, Lee said “slavery as an institution is a moral and political evil…their emancipation will sooner result from the mild and melting influences of Christianity than from the storm and tempest of fiery controversy…we see the course of the final abolition of human slavery is still onward, and give it the aid of our prayers.”
Some people do believe that it is evil to use and abuse nonhumans, you know. I need not see a creature as equal to myself, to accord it rights.
Of course, slavery is possible without any racist element at all.
We should not forget that the mode of slavery most familiar to Americans–black slaves, white owners, with the roles of each immutable–was far from the only model. Many societies through history have had slavery that was not “racially” or ethnically based; slave and owner might be from the same ethnic population, or members of different ethnic populations in a region might have each had the potential to become slave or owner, depending on the vagaries of war and economics. Also, slaves in many systems had the opportunity or expectation of becoming free. (Slaves sometimes were freed in America too, but typically at the whim of an owner, rather than systematically.)
I would say that the owning of slaves by people who were themselves former slaves was not only evident in most (all?) slave societies, but in fact was typical and not particularly interesting to anyone involved.
In our modern perspective, we see slavery as a discrete category of relationship between two or more humans, without any possible redemptive element, utterly incomparable to any “free” system. But through most of human history, I think, slavery–in many different manifestations–were more like a collection of tiered relationships, not manifestly distinct from the many other kinds of relationships two people or groups might have. Being a “slave” of a certain sort, in a certain time and place, might well have been preferable to being a “free man” of another sort, at another time and place, for those involved.