I mean basically, yeah. If they premised the recognition of rights for some but not for others on the idea that others were a lesser race of human, and so not on the same plain of “natural rights” as themselves, then it stands to reason that attacking the premise–or just flat out rejecting it–opens the door to go rectify the foundational error and extend equal rights to all under the very same Constitution.
We work with what we can. This is how constitutional law works. If there is favorable precedent, use it. If there isn’t, explain why existing precedent really supports your position if you look at it a certain way and entertain the possibility that taking a wider view of the underlying issue might undermine current understandings of what that precedent really means.
Fredrick Douglass is the one who tried to sell the Constitution as a pro-freedom document (as opposed to a pro-slavery document) by pointing to the text and offering explanations for how they endorsed abolition so long as one did not entertain the hidden intent of the founders as persuasive. His position, not without merit, was that if they wanted to enshrine slavery into law, and see it protected as a right, and see black people denied the franchise in perpetuity and the like, then the damn cowards should have written it down. But they didn’t, so, as Douglass reasoned, we should give no weight at all to what they might have “really” meant if that didn’t make it into the actual text, particularly as we must consider that the founders were not even unanimous in their position on slavery and in their interpretations of the text.
Honestly, Douglass put forward a pretty good case. But only if you grant that founders’ intent is of minimal consequence in interpreting the text. And of course the Supreme Court in Dred Scott went the other way with that: they leaned hard into the undercurrent of racism prevalent at the time the Constitution was drafted and ratified, and in the decades after, including in the North.
So Douglass wanted to largely ignore the founders’ intent, and the Dred Scott court leaned heavily into their racism as evidence of intent, which they considered (unlike Douglass) to be essential to understanding the text. I am simply trying to make the point that a more favorable result (for the anti-racist crowd) could be achieved by emphasizing the founders’ profound ignorance. Did they really consider that they were subjecting fully equal humans to slavery? Perhaps some of them did, but if they didn’t–if the prevailing view was that they were dealing with a lesser race of man–then it stands to reason they never fully considered the question of whether the Constitution might accommodate slavery as it was actually practiced (humans enslaving fully equal humans), only as it was practiced in their minds (white people enslaving black people, with the presumption that blacks were lesser).
More like, they were morally deficient (or, heck, bankrupt if you prefer) because they didn’t acknowledge the equal humanity of blacks. But at the same time, because they didn’t acknowledge the equal humanity of blacks, they never really contemplated enslavement of fully equal humans.
Or maybe they did. Maybe the only thing keeping the founders from giving the thumbs up to white-on-white chattel slavery was that it had already gone out of style and no one really tried to start it up again in the US under the Constitution.