I have a number of bikes, ranging from one speed (sort of, I’ll esplain below) to 27 speeds. They all have their place.
It is a case of of horses for courses. The single speed is actually a fixed gear with a “dingle” double cog on the back and double chain-ring on the front. So It is actually a two speed, but I have to stop, loosen the rear wheel, manually move the chain over, and retension the chain to shift gears. This means it stays in the high gear unless the situation really demands the low gear to climb a long, steep hill, or deal with a strong headwind. It is also a flip-flop, but I think I have used the freewheel maybe twice in over a year.
The above bike is good for reasonably level routes, in good weather, with no need to carry stuff besides me and maybe what I can fit in a knapsack. A car analogy would be an open, two seat sports car. Adding fenders and racks and otherwise Freding it out would also add weight and make it far less able to climb hills without the option of downshifting. It is pretty fast and fun for the situations where it works.
My main commuter is a 14 speed internally geared beast. It is an excellent transportation bike with wide stout tires that laugh at potholes and goatheads. It has racks front and rear, and I frequently stop and buy groceries on the way home from work. Sometimes I even stop at costco on this machine. It has fenders and will soon have a chaincase so that it can be ridden in the rare rain without maintainance problems. It has dynamo powered lights so that I needn’t worry about getting home before dark. It has a holder for my coffee cup. All this Fredness adds weight, it weighs more than double what the fixie does. and the gears are very useful for hauling that weight up the hills. I put a little over 2000 miles a year on this bike. This is the Subaru hatchback of bikes.
If I need to haul a LOT of chit, then there is the longtailMundo. That has 21 theoretical speeds, though I don’t think I have ever used the granny ring, as the tallest gear is only like 80 inches or so. The granny ring is there if I ever need to haul a lead brick collection up a 10% grade I guess. I do end up shifting a lot, as the gears are handy to get the mass moving even on the level, and brisk acceleration from a stop is needed to get enough speed to make balancing easy with a load. This would be the pickup truck of bikes.
The Tandem has 27 widely spaced gears. (24/34 to 52/11 on 700C wheels) We use every one of those gears over the course of a year, and at least all the front rings on most rides. In this case I chose the ratios so that where the ranges of the front rings overlap, the ratios are interleaved…by dropping to the next larger or smaller chainring I can half-step the ratios when I am anywhere near the middle of the range. This is very useful as the stoker does not ride nearly as much as me. She is not able to spin as fast as I am, so there is a fairly narrow range where I need to keep our cadence in order to get much power at all. Climbing out of the saddle on a Tandem is a skill my stoker and I have yet to master so we rely on the granny ring , and the big meat is for when we flat out fly on the downgrades. This would be the four-door sedan of bikes.
I have a MTB and a roadburner with 21 and 27 speeds respectively. I use the full range on these, though not on every ride. Off-road the lower ranges on the MTB are needed. In the winter, I fit it with studded tires and use it for commuting on bad weather days, and then I use the big ring mostly. On the roadburner, the granny ring is handy to climb in the foothills in NE Albuquerque, and the big meat for descending same. The MTB would be the Jeep of bikes. Not sure what the right analogy for the roadburner would be…maybe a rally car?
There are downsides to too large numbers of gears. You carry the weight of all of them at all times even if you are only using one or a few. Extra sprockets on the rear shift most of the load to the drive-side spokes and make the rear wheel laterally weak due to low tension on the NDS, and poor bracing angles. If you increase the rear triangle spacing for less dish, then you have to widen the cranks so they clear the chainstays, and some folks claim that hurts pedaling efficiency and can cause joint stress. (google Q-factor) The same issue arises when you add front chainrings. Adding more cogs doesn’t add as many useful gears as it might due to cross-chaining. Running narrower chain to reduce Q-factor weakens the chain and increases wear. Wide range gear systems need lots of take-up range, so the rear derailleur becomes bulkier, heavier, and more prone to damage.
On an internal gear hub, adding gears adds weight, complexity, and expense. It may reduce reliability and efficiency as well. There is only so much room available, so compromising will be required. The Rohloff does a good job of achieving all the goals except keeping the price down!
Due to the limitations of the power plant, bicycles have always been an exercise in compromise, and this takes place in a multidimensional space: efficiency vs. weight vs. comfort vs. cost vs. durability vs. strength vs. high maintainance vs. what people want.
As far as how many gears do you really need? The Rohloff 14 speed hub pretty well nails it. Plenty of range, (low enough to spin the back tire on a steep grade, high enough to descend at over 30 mph) and close enough spaced that you seldom wish you could split them. It is heavy and expensive and needs a wide chainline though.