OK, back to the actual OP here.
Yesm, it could have happened. In fact, it nearly did back in the 18th century. However, several things happened, which have been alluded to but not explained.
First, the cotton economoy boomed and brought new “life” into slavery. The cotton gin made labor-intensive short-staple cotton very profitable. That led to new demand for slaves, and as a point of fact, Southern slaveholding was comparitively less brutal than Caribbean or Brazilian slaveholding (the population growth figures unequivicably prove this). So the slave population boom happened. This also meant that people had to seriously confront the concept of what they would do with a population of free, uneducated people of a different race, uncertain national loyalty, and ambiguous cultural background. That was not a trivial question, and essentially was never specifically answered even after the Civil War. It was simply booted down to later generations. (Then it died out as a question entirely until that damn fool Great Society wrecked things up again.)
Second, the more prominent antislavery figures among the Founding Fathers like Franklin and Adams were northerners. Southerners who abandoned slavery weren’t specifically spokesman against it, although they have the moral advantage of putting their money where their mouths were. Antislave movements existed especially in the north, and succeeded there, but they didn’t spread to the south and made few attempts to seriously penetrate the south.
Third, The antislave movement got aggressive. It’s practical a law of human nature than anybody who starts out with moral suasion (anti-smoking, anti-fat, anti-liquor) will end up becoming a tyrannical bastard dedicated to brutalizing their perceived scum-sucking enemies. The South was a little irritating, so the Abolitionists yelled louder. The South didn’t listen, so the Abolitionists got aggressive and angry. Southerners got their feelings hurt and responded in kind, etc. Basically, the American traits of pride and name-calling started to clash in a very, very unhapopy way on both sides. But there was a long time before this when things were happier.
Fourth, it became a political tool. Both free-soil and slave states believed that whichever spread would come to dominate the national life. Combine this with Bleeding Kansas (one of the foulest things ever done by Americans to other Americans), and what was once merely a matter of political businesses became a brutal hot-button issue.