You’re missing a few things:
First, decentralization is itself part of the libertarian ethic, as decentralization allows for more choice. If you absolutely detest your local zoning board, you can move across the city. If you hate the way your city is governed, you can move to another one with more difficulty. If your state becomes tyrannical or embraces policies you don’t like, you can move to another one with even more effort, but you can still do it.
Moving to another country, however, is often not possible for non-professionals, and the cost and disruption of your life is far more dramatic. You can move from one city in your state to another and maintain connections with your family, keep your certifications, have your driver’s license recognized, etc. It gets much more difficult if your only escape from policies you disagree with is to renounce your citizenship and move to a new country.
Second, decentralization acts as a brake on extreme government policies, exactly because it’s so easy to move people and capital. Set your taxes too high, and wealthy people will leave. Oppress gay people, and they’ll find another state to live in. Ultimately, in a country with 50 autonomous states there is a ‘market’ for government. People can go live where the government suits them best. This happens all the time.
Decentralization also allows for more lifestyle experimentation and opportunity for like-minded people to create communities that suit them. Venice California has a population that would be totally unhappy living in Provo Utah, and vice versa. That’s real diversity, which even liberals say they support. I’ve never understood how a philosophy that puts so much emphasis on ‘diversity’ always seems to want to centralize government and make universal rules for everyone.
Allowing 50 states to set their own policies also allows for experimentation and innovation that you can’t get when everything is controlled from a central authority.
To a libertarian, the ultimate in decentralization is the market itself. If we all had to vote on what kind of car we had to drive and what color it should be, we’d either have 51% of the people upset, or we’d be driving beige sedans. Democracy by voting is at best a rough approximation of what individuals actually want. The market allows me to ‘vote’ every time I buy something. But even better, it allows for enough variation that even if 70% want red cars, I can still find a yellow one. If car color was chosen by democratic vote, I’d be screwed.
So no, decentralization is not inimical to libertarianism: It’s essential for it. Your example of a centralized force is something like a benevolent dictator, who retains absolute power so he can simply relinquish it back to the people. The British governor of Hong Kong comes to mind as such, or the monarchs that allowed free trade cities and networks to exist without interference in early pre-renaissance Europe.
In the case of the U.S., the way the system is supposed to work is that the states are left alone to choose their own paths, with a central government that exists in limited fashion as laid out by the Constitution. The Constitution is essentially a document which sets the limits of government, and not the limits of what the people can do. Some parts of it only limit the federal government, while other parts restrict what any government can do - local, state, or federal. For example, a state cannot put you in jail without due process, or take away your right to own a gun or to speak freely and associate with whoever you choose.
So… A libertarian would argue that government is best that governs least, and what government is necessary is best done locally if possible. I’ll tolerate a lot of restrictions from my condo association, somewhat less from my local zoning board, even less from my city government, less still from the state govermment, and least from the federal government. Local government is more likely to be representative of my needs and wants, and is easiest to get out from under if it doesn’t.