I get that flying at lower altitudes causes excess drag - possibly to the point of structural failure if you’re trying to cruise at 500+ MPH somewhere close to sea level. But if flying higher reduces drag, then why stop at 40K feet? There are planes with higher service ceilings, although I know this doesn’t necessarily mean they’re at their best when they’re that high up. Is it just too hard to make a plane that can cruise efficiently/economically at very high altitude and also be well-behaved at sea level?
At some point, the atmosphere is too thin for even the turbines to manage. I think there are also limits between the inside cabin pressure and outside.
Thinner air gets you less drag, but it also gets you less oxygen, which is a necessary ingredient in fuel combustion. Cruising altitudes are set to find a happy medium where fuel combustion is most efficient while maintaining the least amount of drag.
Airplanes that fly higher typically have specialized equipment for dealing with the higher altitude; they also tend to go faster.
There are military planes that can go higher but for commercial aviation, engines designed for those higher altitudes aren’t justified in cost/benefit analysis.
While it does cause drag, air is also what creates the lifting force on the wings. In thinner air, you’d need either bigger wings or stronger engines.
It’s a tradeoff between thinner air resulting in less drag and thinner air (oxygen) resulting in less thrust*. Those aircraft which travel at 60K feet often have a thrust/weight ratio of about 1 whereas commercial jets have a thrust/weight ratio of 0.2-0.3 Thrust-to-weight ratio - Wikipedia They’re also built in a shape which is optimized for speed and maneuverability rather than volumetric carrying capacity.
Giving a much larger thrust/weight ratio to a 747 would be quite costly and fuel inefficient for little gain. Making it more aerodynamic would drastically reduce its passenger capacity. Actually, now that I think about it, there was a commercial jet which had an 18km cruise altitude: The Concorde. Concorde - Wikipedia
Commercial jets are built primarily for large passenger capacity and fuel efficiency whereas those are not major considerations for fighters and spy planes. Commercial jets are flying buses. Fast flying jets are more like motorcycles or racecars. You might give a bus the same power/weight ratio as a motorcycle/racecar but it wouldn’t make much sense economically.
*If you want to get an intuitive understanding of this, play some Kerbal Space Program and experiment with building different types of planes.
High altitude also allows the airplane to fly over a lot of inclement weather which is more comfortable, safer, and may be more fuel efficient.
As for why 35,000-40,000 feet, that is around the height of the tropopause in the middle latitudes where most airlines fly:
Jet engines get more efficient the colder the air gets, that is the hieght where it stops getting colder. So they design the planes for that height.
Sailplanes can do that!
In February 1961, Paul Bikle flew a Schweizer 1-26E glider to record 46,000+ feet in the Lancaster, Ca. area. His son, Hugh Bikle, runs a little vintage aircraft museum at Hollister, Ca., airport, where that glider is on display now.
In February 1986, that record was finally broken by Bob Harris, who flew a glider to 49,000+ feet. Too bad, though, he broke all kinds of rules in the process, wherefore the Soaring Society of America published his feat only in a small footnote somewhere instead of a full feature article, not to mention he got into deep shit trouble with the FAA. His main infraction, among many, was flying above 18,000 into controlled Class A airspace without prior clearance from Air Traffic Control. That’s up in commercial jetliner territory there, so you might really want ATC to be watching out for you up there.
More recently, in Sept. 2017, the Perlan Project flew one of their special-designed high-altitude gliders to 52,000 feet.
It’s fairly common for gliders to be able to fly well above 18,000 feet, but without an ATC clearance, it’s illegal to fly above 17,999. Flights that high are not uncommon. Just a few weeks ago, some people I know (and have flown with) did that. Wish I was with them that day!
Blog of the event, with pics:
Video of excerpts of the flight: 12/16/2017 Wave to 18K Over Watsonville - Full Flight - YouTube (about 30 minutes long).
You gotta dress warm up there! I recently read a blog of a similar flight, in which the pilot(s) discuss how heavily bundled up they had to be! I’ll post an excerpt of that when I find it again.
Here it is. These guys flew an epic 8+ hour flight over the northern California coastal mountains in December, spending some substantial portion between 17,000 and 17,999 feet. In this post, one of them advises how to dress for a flight like that:
How cold is it on average outside the jet at 35,000’?
Why Plane Windows Don’t Roll Down, as Romney Would Like
And there’s the wind chill. 
There is one other factor that hasn’t been touched on yet, the speed of sound. Unless you specifically design for it, the speed of sound limits how fast a commercial jet can fly. Aside from the Concorde, commercial jets are not designed to fly faster than somewhere between M0.8 and M0.9. As you get higher and the air gets colder, the speed of sound reduces. At the same time, due to decreasing density, the stalling speed increases. The increasing stall speed and decreasing speed of sound means that the high speed limit and low speed limit get closer together as you get higher. Additionally, a heavier aircraft (more passengers, more fuel) has a higher stalling speed so the stall speed meets the speed of sound at a lower height. Regardless of the stated aircraft limitations, the real limit may be significantly lower for a given flight.
Factoring all this in, to design a jet to go higher it needs to either go faster than Mach 1 or it needs to have a lower stalling speed.
Another factor is oxygen for breathing. If an airliner loses cabin pressure, the crew and passengers can breathe using masks that supply oxygen at ambient pressure and are not sealed and pressurized. This works because the normal atmosphere is around 20% oxygen, so if you’re at an altitude where there’s something like 1/5th the air pressure, you can breathe 100% oxygen and still get enough.
If the plane goes too high, it gets to a point where the pressure is so low that even 100% oxygen from a simple mask isn’t sufficient. You would need more pressure, which means the crew would have to wear pressurized space suits like a U2 pilot.
In addition to the mechanical concerns, there’s also the time and weight factors. You can’t just miracle yourself 10,000 ft higher. You have to fly there, and that takes time and fuel. There’s a lot of math that goes into it, based on the time to destination, the current weather, the characteristics of the plane, and its current weight and balance. Luckily, that’s what computers are for.
There are people who work for airlines whose sole job is to figure out the most financially savvy way for a plane to fly, given current conditions. Altitude is one of their considerations.
That does not sound right. Colder air is more dense than warmer air, which would raise the speed of sound, not reduce it. The drop in temperature is less than the drop in pressure. The speed of sound is lower because of the lower air pressure.
I always used to think that too, but no. Colder air has a slower speed of sound.
Do airlines save fuel by heating the cabin just enough to keep the passengers from freezing? How cabins heated?
Thank you.
The air entering the cabin normally has to be cooled rather than heated. Aside from the B787, air conditioning systems work by taking hot air from the compressor stages of the engines (“bleed air”), cooling it to a comfortable level, and pumping it into the cabin. The volume of air taken from the engines has far more effect on fuel consumption than the power needed to run the air conditioning packs. You can save fuel by mixing recirculated cabin air with a lower volume of bleed air.
This is pretty much it. Planes fly just below the tropopause, for the most part, or sometimes just above it (the altitude of the tropopause varies by time and location quite a bit).
This altitude is great for fuel efficiency and smooth flight; unfortunately it is also the part of the atmosphere that tends to most favour contrail formation, which means that all those flights are creating lots of high cloud, which traps extra heat. It’s thought that the climate impact of contrail cloudiness is actually considerably greater than the impact of the CO[sub]2[/sub] emissions from aircraft. Cite
Another unfortunate side-effect is the proliferation of chemtrail lunacy on the internet. 