There is apparently a minimum size required for a bird species to be successful. Normal, non-hovering, birds all seem substantially larger than the hummers.
Presumably there are few advantages to being much smaller than a wren and some significant disadvantages.
The disadvantages probably include predation from larger birds, vulnerability to insect attacks (including spider webs), and rodent attacks, and food competition from mice and insects.
But enough rambling, does anyone have any semi-authoritative information?
Hummingbirds are nectarivores (i.e. they consume the nectar of flowers, which use them as carriers for pollination) and as such many of their physical characteristics have similar morphology to insects; in particular, the need to hover in stationary flight places a pretty hard allometric restriction on body mass to wing size and loading. This can’t just be scaled up to any size because as the wings get larger the inertia of the wing and the joint and ligament stresses from loading increase non-linearly (in disproportion to mass) meaning that at the size of something larger than Patagonia gigas that hovers like a hummingbird would be physiologically prohibitive.
The aerodynamic flight modes of the hummingbird have been and continued to be studied extensively both for biological interest and for applications to engineering.
There is actually quite a bit of overlap in weight between hummingbirds and other birds. The smallest passerines, including some flycatchers, kinglets, gnatcatchers, sunbirds, and others, get down to about 5 grams (maybe a bit smaller; those were the lowest weights I could find offhand). This is roughly the same weight as a typical hummingbird here in Panama.
In North America, Ruby-throated Hummingbirds weigh about 3 gm. The smallest hummingbird, the Bee Hummingbird of Cuba, gets down to about 1.6 gm. The largert hummingbird, the Giant Hummingbird of the Andes, weighs a whopping 23 gm, about as much as a Starling.
So although the smallest hummingbirds are smaller than any other birds, there are plenty of birds that are just as small as the average hummingbird. I would guess that hummingbirds can get so tiny because of two factors: 1) their nectar diet gives them a very high-energy food source; and 2) their unique flight makes them maneuverable enough that they have access to flowers not available to other birds.
Another potential disadvantage for small warm-blooded creatures is heat loss; the rate at which a creature loses heat is inversely proportional to its size. Hummingbirds have found a particularly energy-rich food source to compensate for this effect, but insectivores like flycatchers or kinglets might not be able to find enough food to stay warm if they got much smaller.
All hummingbirds eat some insects, spiders, and other small arthropods. Flower nectar lacks protein and vitamins (though some has amino acids), and they need to supplement their diets with non-sugary foods. In particular, they need to feed growing young with plenty of insects.