You seem to be forgetting a few things, ranging from the barbarians moving in to which people were most likely to end up on top under the new management… the immense majority of the Roman Empire’s inhabitants were not citizens.
In high school, I had an exam with the essay question something like “Defend the premise that Rome fell because of the angst of a jaded people.” I forget what I said…
The standard empire model is that the new/outlying territories feed the center/motherland with raw materials and take the manufactured products. this drives the engines of commerce, and the central government gets a good rake-off from that. Eventually, however, the outlying areas are also going to develop their own commerce and industries. They do not need the center’s industry and resent feeding it with their resources and taxes. (Sounds like the 13 colonies).
Sometimes the ruling center uses law and force to prevent development, which will simply encourage more resentment and rebellion. Sometimes remote territories don’t care about problems far away - Constantinople really did not care that Gaul and eventually Rome were overrun.
Sometimes the central ruling area allows the other territories to become developed, and try to create a mutual trade zone, like the mid-era Roman Empire or the USA. However, that doesn’t stop a disintegration like the barbarian invasions that Constantinople did not care about. It also does not stop the economic decline as in the USA when the industries left, or moved south and west to leave the original north-east as the “rust belt”. The saving grace for the USA is the level of mobility means that most people identify as “American”, not “Californian” or “Georgian”; so the whole stays together, even though some areas appear to resent paying for the perceived costs of others. (Red vs. Blue, anyone?)
It’s a nitpick ( which is why we are all here of course 
), but actually the majority probably were. Caracalla’s edict in 212 extended citizenship to all free adult men ( and the corresponding lesser rights of Roman women to all free adult women ) throughout the empire. The unfree would have been a sizeable chunk of population, but almost certainly nowhere near a majority. Especially once Christianity became established, which in this case seems to have led to a decline in slavery in the empire.
Of course your larger point still stands. Free or unfree, late period peasantry which made up the great bulk of the population had little real stake in the empire. The Roman empire was supported by and existed to protect the rights of the tiny minority of the wealthy landowning class. Once the empire was penetrated and large areas were effectively lost to the central control most of those landowners had to come to an agreement with the new local power, whomever they may be, in order to safeguard their property. When your wealth depends on owning land you can’t just pick up and leave when the Vandals ( for example ) occupy the area. At that point you’re paying your taxes to your new benefactors/overlords who derive wealth and power from their new tax base that is now denied to the center. For the WRE it became a death spiral as the center could no longer afford the armies to win back the periphery and the new peripheral powers grow wealthier and better able to resist pressure from the center.
Meanwhile the poor bloody peasant, citizen or not, shrugs their shoulders and keep on plowing their fields and hope they aren’t butchered or dispossessed in some random act of violence.
Paul Kennedy wrote a well-received book The Rise and Fall of the Great Powers, which examined different examples of this throughout the last 500 years. He attributed it to the thesis that wealth and power are relative, and that therefore even a Great Power that did everything right could still find itself declining due to military and economic competition that simply hadn’t existed previously, and therefore finding itself relatively weaker and poorer.
One point I’ve heard is that the “barbarians” kept getting more effective. When Rome first began expanding it was conquering lands where the locals were divided up into small tribes. The combination of seeing how much more effective Roman organization was and feeling the pressure from Roman expansion caused these locals to revise their political systems and band together into larger groups like kingdoms. And when these kingdoms got big enough they were able to conquer Roman territory.
Yes, it has been argued ( based on archaeology ), that the Germanic population in particular, presumably at least in part due to exposure to the Romans, were by the 4th century larger, wealthier, more economically productive and more politically sophisticated and hiearchically stratified than it had been in the 1rst.
It was also socially and politically fluid. In the face of Roman aggressive pressure the Visigoths were basically created by Alaric from the amalgamation of three disparate Goth groups around 408 - forging a new super-tribal entity. The Ostrogoths of Theoderic the Great, however ancient the name ( much argued about ) basically were created the same way on the spot. Clovis created the unitary Frankish state, formerly seemingly a loose confederacy of sorts, by simply conquering six other Frankish groups and adding them to his own. Similarly with Gaiseric’s Vandals ( Hasding Vandals + shattered remnants of the Siling Vandals and the Indo-Iranian Alans ).
As a consequence these new super-tribes were arguably far more formidable militarily than past ‘barbarian’ enemies. Still only part of the story of course, but between the pressure of Persian Sasanians in the Near East, these new tribal entities in Europe, and the westward moving Huns the Romans had a tougher row to hoe by the time the 5th century rolled around.
Thank you all. Very helpful.
davidmich