The super cheap RC helicopters I’ve played with require manual control of yaw (and quite a bit of coordination to do so I must say).
It has already been explained that they don’t use capacitors, but even the best capacitors nowadays are still far inferior to batteries in terms of energy storage. Not only that, most small ultracapacitors have a very high internal resistance (ohms vs milliohms for batteries; see the one Dr. Strangelove links to, which has 4 ohms of ESR and in any case 1 farad stores only 3.645 J of energy at 2.7 volts, or 3.6 watts for 1 second; a 3.7 volt 130 mAh Li-ion cell (from Wesley Clark’s example) can deliver that much power for almost 8 minutes - 475 times longer). You also have to deal with the fact that the voltage drops continuously (exponentially if you try to draw constant power since energy is 1/2(CV^2), so you need a DC/DC converter to deal with that (that also assumes you can discharge it completely to 0 volts, although at 1 volt you have only 13.7% of the energy at 2.7 volts left, and you could use two in series).
For another perspective, according to HowStuffWorks you would need more than 10,000 farads to store as much energy as a AA battery.
I think the big innovation has been the Lithium Polymer batteries, they are powerful and light. They would keep my $180 Blade CX helicopter flying for 10 minutes or so. Only problem is LiPo batteries are potential fire hazards (youtube video of intentional LiPo puncture.)
Oh my goodness, a VertiBird! I had one of those when I was about twelve years old. I loved it – it’s one of my best-remembered toys of my childhood. But boy, was it hard to control.
You’ll have to define “toy”. If it’s the little buggers we have now, then Li-Po batteries and tiny motors are key.
If you just go with remote control, they’ve been around since the early 1970’s at least. I had a couple in 1979 or so. There were certainly more planes than helos at the flying field back then, mostly because of the expense. You could get a plane and engine for under a hundred bucks, but a helo would be closer to a thousand all in.
I say we send this guyinto Afghanistan or wherever needed. He can cruise a camera sled through town, darting aroung like a dragonfly. Get live pictures, and they’ll never shoot him down. 
I love the smell of glow fuel in the morning. Smells like…skipping school!
Wow. I remember those too. My brother and I had one each - you could set them up so the helicopter flight paths overlapped, just a little…
I think that was it - I remember the space capsule that you rescued.
I’m thinking that with the palm-copters, the innovation that makes it feasable is putting the entirety of the receiver and control circuitry on a spec of silicon under a dot of epoxy. Several orders of magnitude of savings on weight and energy consumption.