It’s possible that the United States could have split into two countries. One with at least nine states where the Constitution was in effect. And another smaller country where the Articles of Confederation were still in effect.
People at the time recognized that they needed Virginia and New York in the country to make it viable. So if those two states hadn’t joined the Constitution country, it probably would have been accepted that the Constitution wasn’t going to work even if it had been ratified by nine other states. There most likely would have been another convention to write a new version of a constitution or a revision of the Articles of Confederation that Virginia and New York would agree to.
If it had been Rhode Island or Massachusetts, they probably would have decided to go on without them. The United States would have dropped down to eleven states and Massachusetts and Rhode Island would have been a separate country.
Most people would have seen the Constitutional United States as the future. Vermont, which was on the verge of statehood, most likely would have joined it rather than join up with Massachusetts and Rhode Island. And all other future states like Kentucky, Tennessee, Ohio, Alabama, and Maine would have been certain to join the big country which they bordered rather than the small Confederation of Two.