Airplane simulators: How can they be trusted?

FWIW: In the abnormal situations I’ve experienced in real life, I’ve felt well prepared and mostly reacted correctly.

Not long ago, a colleague experienced his first engine failure in a jet. He was surprised at how he not only remained calm, but was actively trying to mentor his first officer in this situation. We drill on that stuff frequently and in worst-case scenarios. So when the real thing happened it was relatively benign and he felt very comfortable handling it.

I imagine that’s the goal of any training. Buzz Aldrin talked about this in regard to the moon landings. He essentially said that in an extraordinary situation they had to simply revert to how they were trained. Of course, this assumes the training was good.

Add a few flashing lights, and some crewmates keeling over dead, and it’s the Kobayashi Maru test.

I’ve flown Cherokees as a student pilot, and I had the same experience as you did trying to land a large jet airliner in Microsoft Flight Simulator. Not a hope!

Two magic words apply here: “stabilized approach”.

Small airplanes like Cherokees and Cessnas are analogous to sports cars. Large jets are analogous to much, much larger vehicles, somewhat more like ships than cars.

So the idea is that if you’re to have any hope of a successful landing, you have to be configured for it in the early stage of the final approach. Once in a stabilized approach, in theory if you do nothing at all you’ll touch down on the appropriate spot on the runway. IRL, small adjustments are necessary, but only small ones.

One time I was looking out the window as the jetliner I was in was approaching for a landing, and suddenly the speed brake came up on the wing for a short period, causing the airplane to lose altitude rapidly for a few seconds. Obviously the pilot had screwed up and realized he was too high.

There is of course a concept of a stabilized approach even in small planes, but it’s more of an informal “fly by the seat of your pants”. AIUI with large aircraft it’s much more disciplined and methodical, because your ability to make rapid corrections is limited.

You seem to be under the misapprehension that if a simulator does not naturally match real-world failure states, it cannot accurately depict real-world failure states. This is false.

I am not a pilot, but I did supervise and conduct training on a nuclear propulsion plant. We have simulators for those as well, and they have reached a sufficiently advanced point that, in the Navy at least, any training performed on a simulator can be treated as if it was performed in the actual propulsion plant (for “maneuvering” or “enclosed operating station” watschstanders, anyway).

The way you train people to deal with equipment failures isn’t by having them operate the plant (or a simulator) long enough to see every possible fail state arise naturally. Indeed, the expectation is that certain fail states should never arise naturally. You do it by either (a) simulating failure conditions in a real plant (such as by setting up a fake meter readout next to the actual meter) or (b) putting someone in a simulator where you can punch numbers or play out a pre-planned scenario in a computer and have the “actual” (simulator) meters actually display the abnormal or emergency conditions.

In this respect, simulators can actually be much more realistic, because it eliminates the need to have a member of training staff stand up and out an obviously fake read-out next to the real meter (gee, do you think that might be a sign something is up?—actually, we intentionally displayed red herrings specifically to reduce that risk). Plus, you can “actually” do things in a simulator that you wouldn’t dare do in the real world. The aviation equivalent would, I suppose, be like a wheels-up crash-landing in a field. I don’t know if simulators are designed to do that, but I can pretty much guarantee you that no flight school is going to expect students to actually crash land an actual plane wheels-up in a field.

So to the example you used, while it might be true that a simulator might not naturally have pilots flying around with certain transmitters off when they should be on, if someone wanted to run that simulation—for whatever training value it might add—far better to do it in a simulator, I think, than out in the real world over an actual densely populated area.

That’s a fascinating insight, thank you. Somewhat surprising, but fascinating.

Many years ago I visited a facility that made full-motion simulators for airlines and for the military of various countries. Wow, what a place! I would have loved to have had a chance to play in one but, alas, that was not in the cards.

One thing I remember is that back in those days, Ethernet was still 10 Mb/s. They were using powerful computers quite capable of generating the appropriate imagery and physical feedback, but 10 Mb/s just didn’t cut it. So what they had instead was vast thick bundles of Ethernet cables tied together going from the computers to all the image generators and actuators.

One of the impressive projects they were working on was a military jet simulator for the Israeli government. Rather than having imagery projected on front and side windows, they had somehow contrived to have the simulator produce 3D-like imagery across the entire dome of the fighter jet.

My only thought at the time was, “these are the greatest toys in the world!” :grinning_face:

For those of us who aren’t aspiring pilots, you can still experience the magic of simulators… a few museums around the country have interactive hydraulic simulators like Products – Pulseworks (at the Smithsonian in DC and the Museum of Flight in Seattle, among other places).

You can fly a fighter jet in them, do rolls (it’ll actually invert you), etc. They’re not meant for training, obviously, and you only get like 5 minutes in them at a time, but they’re loads of fun as toys. Wish I had one at home!

I’ve been staying out of this thread because I can’t wrap my mind around what the OP’s comments can possibly mean.

What do they think is going on in this large segment of highly regulated aviation?

(fist bump from another Navy Nuke!)

On the simpler question of the physical realism of simulators, consider that Formula 1 racing teams, in addition to spending hundreds of millions of dollars to build and race the most advanced cars in the world, also spend tens of millions developing and operating highly advanced simulators for their drivers.

ETA: They obviously wouldn’t do that if it didn’t provide a measurable benefit to the performance of their drivers.

Mechanically, the technology is a much scaled-down version of aircraft sims, but since driving an F1 car around a track involves hundreds of split-second inputs, I suspect the overall complexity of the system is not as much reduced as one might expect. A much simpler cockpit and vehicle to simulate, of course, but with driver inputs and vehicle reactions that are much, much more rapid and subtle. And no autopilot!

Here’s a ten-minute video on developing the simulator for F1’s newest team.

Perhaps I wasn’t clear enough at the start. If flight simulators are to be useful, they have to emulate the flight experience as closely as possible. If they do not do this, what the sim users will learn will be of less use in a real experience.

Therefore, the question is, “How can we be sure (or how much can we be sure) that the sims match the real thing?”

The answer seems to be, “We know they are close enough to be useful.”

My wife has a cousin who lectured at a UK University with an Aeronautics department. One of the things that this department did was aircraft exit simulations. They mocked up the aircraft in a hanger, and paid a bunch of students and locals to act as passengers and check the evacuation procedures several times. A small additional financial incentive was used to encourage “panic”.

Airbus commissioned this to test the exit procedures for the upcoming A380.This required a much larger group of passengers, and tested exiting from both the lower and upper decks. For the first time in the program, participants refused to perform additional exits after the first exit off the upper deck. The emergency slide was safe, but the perception was that it was too steep. Airbus had to redesign the upper deck exits to adjust for passenger perception.

On a similar note, the swimming pool I frequent provides facilities for airline and ferry staff training. Every few months, an large group of cabin crew and flight crew turn up, don overalls over their swimwear, and practice deploying and boarding a liferaft in one of the pools.

Flight and Flight Simulators essentially rely on two things. Physics and systems. Both are, for the most part, extremely well understood.

Or — and this is something I just learned this from this thread — sometimes they can even be worse (as in more difficult) than reality itself, thereby overtraining pilots to handle situations that turn out to be not so bad in real life.

That seems like a good thing to me. Better to train on 150% difficulty than 90%.

my (somewhat educated) guess:

Shouldn’t matter much, as … given the chaotic nature of airflow, every (real life) stall will ALSO be slightly different, too, prob. depending on height/temp/humidity/windconditions/whatnot …

so, the sim-stall will be one-more-realistic-stall for any given plane…

Here’s a video from Mentour Pilot, a respected YouTube commentator and professional pilot, that touches on this question.

You can watch the video or not, as you please, but the summary is that on May 26, 1991 a Boeing 767 accidentally deployed the thrust reverser on one engine while in flight – something that Boeing had not considered to ever be possible due to multiple safeguards. The accident investigation revealed the unexpected causes of what turned out to be a fatal crash, but here’s the relevant point from Petter Hornfledt’s commentary (emphasis mine):

… but there was still an issue of why the aircraft couldn’t be controlled – the pilots should have been able to handle it. Lauda Air spent on a lot of resources on this and Nicki Laudi himself was deeply involved in the investigation work. Among other things they tried to replicate the event in a flight simulator and among other things it showed that this event should have been controllable. But it turned out the the simulators just didn’t have the performance parameters to accurately simulate the handling characteristics in this incident.

The video goes on to describe how Boeing then did wind tunnel testing and eventually upgraded the simulators with the appropriate parameters. It then turned out that, yes, the situation could in theory have been managed, but only if the pilots had reacted almost immediately – within 4 to 6 seconds of the mishap.

The full video:

Not a pilot, but a mech engineer who worked for a while on human spaceflight hardware. My systems weren’t flight systems, but the philosophy of design practices was discussed. We did spacewalk equipment - a unique environment hard to replicate on Earth.

Simulators didn’t have the ability to completely replicate all aspects of the experience. Rather, a combination of methods were used to train aspects of the experience to provide cumulative experience that have familiarity if not direct experience.

For example, weightlessness can be simulated by suspending horizontally by cables, but that only reorients gravity out of the plane of study. This technique was used to teach about walking on the surface of the moon.

A parabolic flight on an airplane can provide short periods of weightlessness, but only about 90 secs followed by 90 secs of 2 gs. Vomit comet indeed.

Large pools can use neutral buoyancy to assist in movement of large objects and for geometry assessments in spacesuits, but drag is significant and weight still exists.

Thermal-vacuum chambers can provide pressure and temperature practice, but at full weight.

Each of these experiences is aimed at part of the experience, but not the whole thing.

Flight simulators have extensive design going in to provide the type of movement and system operations to match a real event. It takes extensive design, and testing of the simulation systems to show how they perform and measure the results.

Yes. I participated in failure analysis on some of our equipment. There’s the proximate cause, which is the immediate trigger. Then there is the root cause, the underlying issues that create the proximate cause.

Design for reliability uses the principle of fault tolerance - how many independent single failures can the system tolerate and function or fail safe.

Still, all of the root cause analyses I participated in showed the root cause was a combination of situations and factors - almost never one single factor.

A couple of responses here. First, anyone who has played an engaging video game can experience the amping up of emotional response and even adrenaline kick. While it may not reach the full level of a real situation, it still gives some idea.

Second, my karate training includes self-defense training. One significant difference between classroom and reality is the ability to be mentally prepared and physically warmed up.

However, drills in technique are used to internalize the physical reactions so they don’t require concentration. But we also have techniques to give some experience of the surprise and unexpected nature.

One other feature is that panic often comes from not knowing how to respond rather than thinking about the severity in the moment. Practicing the response provides a confidence during the actual situation.

I’ve had one situation of an actual physical confrontation. It was triggered by a temper-tantrum by an anti-masker during covid. Even though I was frustrated by him, my brain stayed in analytical mode the whole time. And the adrenaline made it feel like time slowed down. I had multiple thought processes all simultaneously, any one of which would normally take all my primary attention. I had the ability to see what was coming and how to react while thinking about which actions I should take.

Adrenaline can cause panic, or it can cause increased reflexes and faster brain processing. Drilling the situations mean the possibilities have already been considered and practiced, which provides a familiarity that means response is just one more repetition.

I’m not sure how a “stall mode” would be different than modeling a complex system replicating individual air currents and physical dynamics with intensive numerical computations to somehow create what an actual wing would be doing. We know physically how stall behaves. Program that directly.

Are computers fast enough now to do that in real time (simulating the fluid dynamics of a stall)?

I know nothing about this, just read How X-Plane Works | X-Plane (which talks about something called Blade Element Theory) and https://www.avsim.com/forums/topic/656642-cfd-better-than-blade-element-for-flight-simulation/ , which discusses various ways sims try to approximate airflow over different frames and airfoils.

I think point is that “stall mode” is just a classical representation of a stall and not necessarily how the real aircraft would behave in a stall. If they could accurately model the underlying physics then no one would ever need to do flight tests.

Simulators get tested periodically for fidelity, meaning they must be checked to see that they perform near enough to the aircraft to be useful.

In your OP I think you are assuming that the simulator is fed a bunch of physics data and then it simulates flying the aircraft based on that. The reality is more that the simulator has a set of data to ensure that it behaves like the real aircraft in certain conditions. It doesn’t matter if it behaves differently in unforeseen circumstances because it is not intended to be used “outside the box”. If you overspeed it by 40 knots, then put the gear out, it’s not necessarily going to behave in a realistic way, and that’s ok.

I think that’s a different kind of simulation. You are talking about design development, trying to simulate the plane’s behavior just computationally. There are other tools for that - wind tunnel data, finite element analysis, etc.

The simulators under discussion are flight simulators that are training tools for pilots. They are intended to teach pilots what flight feels like, how systems respond, proper procedure, etc.

Flight simulators are developed after the flight behavior for the design is understood.

Modelling turbulent flow is one of the grand challenge problems.* But just because perfect understanding is hard, it doesn’t mean that at the scale of an aircraft you can’t derive approximations that are essentially indistinguishable from reality for the purposes of flight dynamics. Particularly at subsonic regimes.

At particularly nasty edges of the envelope things are likely not going to be as accurate, but you may need to be creative to come up with them.

Mostly aircraft are not flying with significant turbulent flows. That is more a part of their plummeting behaviour.

Modelling and wind tunnel testing of even some pretty evil failures will have been done early in aircraft development. Loss of bits of the aircraft, departed engines, ice, undercarriage misbehaving. You need to be able to show that the thing will still fly. And have a story about how it will fly safely and land. Then you get test flight data. They don’t just test happy paths. This feeds sims.

A sim won’t be running CFD models. Rather empirical parameterised models that predict behaviour in real time based on a slew of measured and computed behaviour.

* one of my favourite quotes. From Sir Horace Lamb.

I am an old man now, and when I die and go to heaven there are two matters on which I hope for enlightenment. One is quantum electrodynamics, and the other is the turbulent motion of fluids. And about the former I am rather optimistic.