You might find them a bit loose in the crotch.
That’s because it’s a stupid analogy that has no bearing on economics, urban development, real estate, the realities of local politics and fashion. It’s clearly absurd to have no restrictions on what can be built. There are other considerations besides who has the most money. Health, safety, pollution, infrastructure demands, economics, the general aesthetics and character of the neighborhood are just as important.
Technically, they always have the right to complain. But if someone wants to put apartments in a nearby area that is zoned for apartments, you just have to know you’ll probably lose that argument. Of course, that also depends on how specific the zoning code is.