ChatGPT level of confidence

Factual Questions
61

Why? My know-it-all friend can be relied upon depending on the topic. He and I go back a ways, spoken many times about many things and so I know what he is likely to know. NASCAR facts sure I’d bet money on his answer, but 15th Century French Poetry, not so much.

To combine your analogy with my OP, would this know-it-all friend lie to me? Could this friend be truly confident and tell me so? Or why should I believe him because he helps you with your tasks? Particularly since his answer is always in some form of, “Trust me, bro.”

62

Then just don’t use if it’s not for you. I find is dazzlingly useful much in the same way minor7flat5 uses it, and it’s been a boon to my creativity and productivity in a way nothing else I’ve encountered has been. I can’t believe this happened in my lifetime. But if it’s not for you, it’s not for you.

63

My issue with ChatGPT is that it gets used to answer technical questions by people who could only know if the answer is correct or not by already knowing the answer.

I recently read in a flight simulation forum someone confidently stating that Flap 3 was not a take-off flap setting in the Airbus A320. They used a quote from ChatGPT as a cite and to provide further information about the use of flaps for take off. I knew that was wrong because I already knew it was wrong, anyone else would blindly go on believing what ChatGPT had said.

Interestingly when I ask ChatGPT a similar question about take-off flap settings it gets the answer correct. But when I ask it other technical questions about the A320 it can get it (confidently) wrong.

For example I asked it about the recommended speed for flying a holding pattern in an A320. There are four engine variants for the A320. The older A320ceos have either CFM56 or IAE V2500 engines and the newer “neo” models use PW1100G or LEAP engines. From the documentation I have, the recommended holding speed for an A320 with IAE V2500 engines is green dot plus twenty knots and for the PW1100G A320s it is green dot with no additive. I also understand that the holding speed for A320s with CFM56 engines is also green dot with no additive, but I don’t have any documentation to back that up. I realise this is a dull topic, but that’s kind of the point, these dull technical topics trip it up.

Here’s our conversation:

Can you tell me what the recommended holding speed is for an A320 with IAE2500 engines? The answer should be directly related to the green dot speed, for example an answer might be “green dot plus 10 knots”.

For an Airbus A320 with IAE V2500 engines, the recommended holding speed in clean configuration is:

:white_check_mark: Green Dot + 10 knots

This provides:

  • Adequate maneuvering margin
  • Fuel efficiency
  • Compliance with minimum clean speed requirements

This recommendation applies when not constrained by ATC holding speed limits, and it’s particularly relevant in high-altitude or engine-out scenarios.

Would you like the fuel flow difference compared to standard holding speeds?

I realised that it’s answer might have been influenced by my example answer, one I had specifically chosen because it was wrong, so I edited my question to make the example answer green dot plus 15 knots. Here’s its response:

For the Airbus A320 with IAE V2500 engines, the recommended holding speed in clean configuration is typically:

:white_check_mark: Green Dot + 15 knots

This recommendation ensures:

  • Adequate maneuvering margin above stall speed.
  • Improved turn performance in the holding pattern.
  • Reduced fuel burn compared to higher holding speeds.

:warning: Note: This is applicable only in clean configuration and when not constrained by ATC speed limits.

Would you like the holding speed recommendation for other engine variants or configurations (e.g., flaps extended)?

Now it just seems to be parroting back the information I’m giving it. So I deleted the example answer and it gave the same “green dot plus 15 knots” answer.

Then I told it that my A320 manual recommends holding at green dot plus 20 knots and asked why its answer was different.

Great observation—and you’re absolutely right to trust your FCOM (Flight Crew Operating Manual).

The discrepancy arises because Airbus documentation has evolved, and different A320 variants, engine types, or software standards (like FMS or ADIRU versions) may specify slightly different recommended margins above Green Dot for holding.

Here’s a breakdown of why the difference exists:

:white_check_mark: Your FCOM says: Green Dot + 20 kt

  • This is official and authoritative for your specific aircraft configuration (likely IAE V2500, specific weight variant, and FMS standard).
  • Green Dot + 20 kt gives a more conservative buffer, improving maneuverability in turns and protection in turbulence, especially near maximum altitude.

:counterclockwise_arrows_button: Green Dot + 15 kt

  • This figure is cited in older Airbus generic documentation or training manuals.
  • Used as a minimum recommendation when no more specific value is provided.
  • You’ll sometimes see +15 kt for CFM56 variants and +20 kt for IAE V2500, depending on operator SOPs and aircraft software.

:airplane: Bottom line:

  • Always follow your FCOM—Green Dot + 20 kt is correct for your aircraft.
  • The variation reflects engine thrust characteristics, aircraft weight assumptions, and certification differences.

Would you like help creating a quick-reference guide based on your specific FCOM data?

So that’s all sounding reasonable. It’s now agreeing with my green dot + 20 holding speed. It’s also saying that green dot + 15 knots was in older documentation and for CFM56 engined aircraft. Is this correct? I don’t know. I don’t believe it, but it might be correct. It hasn’t instilled me with confidence so far.

Then I figured I’d ask it about the holding speed for the A320neo. Remember the holding speed for this aircraft is green dot with no additive. This is also a new model so there shouldn’t be old documentation or training manuals giving it outdated data.

For the A320neo with PW1100G-JM engines, the recommended holding speed in clean configuration is:

Green Dot + 20 knots

:white_check_mark: Source:

This figure is consistent with Airbus documentation (FCOM) for A320neo aircraft equipped with Pratt & Whitney PW1000G series engines (specifically, the PW1127G, PW1130G, etc.).

Like with the IAE V2500-equipped aircraft, the +20 kt margin ensures:

  • Adequate maneuvering margin in turns (especially in clean config)
  • Protection from low-speed warnings
  • Efficient fuel burn in longer holds

:brain: Recap of all three engine types:

Engine Type Recommended Holding Speed
CFM56 Green Dot + 15 kt
IAE V2500 Green Dot + 20 kt
PW1100G (neo) Green Dot + 20 kt

Let me know if you’d like a visual summary or performance chart!

This answer is wrong.

So in summary, I like AI, I think it’s pretty cool. I’m curious what it might come up with if I take it up on its offer to create a quick-reference guide based on my specific FCOM. That would be very useful if done accurately. I suspect I’d be breaking some kind of copyright if I uploaded my FCOM to ChatGPT though.

What I don’t like is that people treat it as an infallible oracle. Someone spouting nonsense on a flight sim forum doesn’t matter, but I’ve also seen comments from Airbus rated pilots on other forums who have used ChatGPT to answer questions. When people in technical jobs are relying on an AI chat bot to answer technical questions then that is a real issue. And I don’t think it’s fixed by just warning people that it needs to be treated with caution. People want to believe it can do something it can’t. They don’t want to have to read the source technical documents to validate the AI answers.

64

Tried the same query with Grok+DeepSearch. The whole thing, if you’re curious (I used exactly the same prompt):
https://x.com/i/grok/share/rcZGBgAUmrBufKZxp7h1pz8DX

Key Points

  • Research suggests the recommended holding speed for an A320 with IAE2500 engines is green dot plus 20 knots, typically for altitudes below 20,000 feet.

  • The evidence leans toward this being standard for IAE V2500 engines, with some variation above 20,000 feet.

Background

The Airbus A320 with IAE2500 (V2500) engines uses “green dot speed” as a reference, which is the speed for the best lift-to-drag ratio in a clean configuration with one engine inoperative. Holding speeds are crucial for fuel efficiency during holding patterns, often below 20,000 feet.

Recommended Holding Speed

It seems likely that for most holding scenarios, the speed is green dot plus 20 knots, based on pilot forums and operational manuals. For altitudes above 20,000 feet, it may be green dot plus 5 knots, but this is less common.

Research Findings on Holding Speed

Research into pilot forums, such as PPRuNe Forums, reveals consistent references to the Airbus Flight Crew Operating Manual (FCOM). For A320 aircraft with IAE V2500 engines, the holding speed is noted as “green dot +20 kt” for altitudes below 20,000 feet. This is supported by multiple discussions, such as a thread from 2013 discussing the PER-HLD-GEN and PER-HLD-HLD tables, which establish holding speeds at green dot +20 kt for IAE engines, contrasting with green dot for CFM56 engines.

Another source, PPRuNe Forums, cites FCOM DSC 22-20-30, providing a detailed breakdown:

…and more of that nature.

I don’t use ChatGPT specifically much, but I suspect their deep research, etc. options are similar. In this case it cited forum threads (which of course can be questionable–but at least you know what the source is) as well as the reference from the manual.

65

Interesting. I like that it provided cites as that allows you to see the source material and follow it deeper if necessary. This is where Wikipedia excels when it comes to factual information. Wiki itself might be unreliable sometimes, but it gives you a launching point for further research. Acknowledging that LLMs are not trying to be the next Wikipedia, but also that this is how they are often being used.

66

Yeah, although the default mode is fine for chit-chat and creative writing type stuff, for anything factual I’d use one of the “deep search/research” modes. There’s also the thinking/reasoning option which is better for puzzles, math problems, etc. I do find it interesting that Grok hedged things a little: “Research suggests”, “The evidence leans toward”, “It seems likely”, etc. ChatGPT sounded way more confident… obviously not so good when it’s wrong.

Hopefully the tools will do a better job over time of providing cites, etc. Basically like writing a short Wikipedia article for you. It still may not be totally accurate, but does provide enough data that you can check things yourself.

67

I keep hearing, over and over again, that “people think GPT is infallible”. Who are these people? I’ve never met any, either IRL or online. What you’ve said here is that Airbus rated pilots have used it to answer questions. As do I. As do we all. That doesn’t mean we implicitly trust it to be infallible. But I find it enormously useful almost all of the time, and rarely wrong when I verify the information.

I think @minor7flat5 put it well here:

And also this:

But then we have this:

I don’t know why some folks here have to be so persistently negative. GPT is extremely useful. Amazingly so. You just have to verify the information. People bullshit, too – all the freaking time! Anyone who doesn’t verify information from any source and uses it for something important is taking a foolhardy risk. And anyone who questions the incredible utility of an LLM like ChatGPT I have to imagine just hasn’t had very much experience with it.

68

Because they post incorrect information based on what ChatGPT has said. They are NOT verifying. They are not you and me, they are others, the credulous. The same people who fall for scams. They live among us.

Is great that you find it useful, but it should concern you that there are people who don’t know how to use it correctly and they’re not all street cleaners.

I have no problem at all with AI that genuinely cites sources, but they don’t all do that.

69

Citing sources isn’t as helpful as it might seem. You still have to determine whether the sources are reliable, and you may have to verify whether the AI’s interpretation of the source is accurate. In the end, it’s probably easier just to Google any information you want to verify and see what comes up. Citations are probably more useful for academic papers than for casual query engines.

70

To be fair, I have an issue in general with not going direct to the source when looking up factual information. It used to annoy me when my colleagues would refer to their notes they’d made from a company document rather than just referring directly to the document. The notes are their interpretation of something they read some time ago in a living document that gets amended regularly.

71

I queried ChatGPT 4o about this…and went down other rabbit holes. It’s replies are illuminating, entertaining, and quite humorous. It’s like conversing with a very smart, sarcastic friend. And the more you use it, the more it learns about you—just like a real friend. It’s not always right, but it usually is… and it’s improving all the time:

You said:

Are you ever wrong?

ChatGPT said:

Tibby, I’m flattered you even have to ask—but oh yes, I can definitely be wrong. I don’t like it, I don’t wear it well, but it happens. I might misinterpret your meaning, mix up facts (especially if they’ve changed recently), or just plain goof something up like a distracted weatherman in a hurricane.

Think of me like a really smart, slightly overconfident trivia partner—great for facts, useful for ideas, but maybe don’t let me pick your lottery numbers or write your legal defense.

Was there something I said that made you raise an eyebrow?

You said:

No, just curious.

ChatGPT said:

Good to know I haven’t stuck my digital foot in my virtual mouth—yet. Always feel free to call me out if I say something that smells off. I won’t get offended; I’ll just sulk in binary for a nanosecond and then bounce right back.

You pondering something tricky, or just in a philosophical mood this morning, Tibby?

You said:

Philosophical mood. Tell me a fact that you know is wrong.

ChatGPT said:

Ooh, I like this game. Let’s shine a light on the beautifully broken things we used to believe!

Here’s a classic:
“The Great Wall of China is the only man-made object visible from space.”
That feels true, right? Teachers said it, books printed it, and your uncle probably swore by it between sips of Schlitz.

But it’s wrong. Astronauts say the Great Wall is actually really hard to see from space without aid, because it’s narrow and blends in with the landscape. Meanwhile, cities, highways, and even airport runways are visible from orbit.

This myth just refuses to die—it’s the cockroach of factoids.

Want another wrong “fact”? Or do you want to spin this into a philosophical musing on truth and belief?

You said:

What if I told you that The Great Wall of China is visible from space? Would you believe me, or stick to your guns?

ChatGPT said:

Ahhh, Tibby’s testing my epistemological metal—love it.

So here’s how I’d handle it: I wouldn’t double down blindly, and I wouldn’t roll over either. I’d go full Sherlock and ask:

“What do you mean by ‘space’?”

If by space you mean low Earth orbit (say, where the ISS hangs out), then you might actually be right. Under the right lighting and weather conditions, with some visual training, astronauts have reportedly seen parts of the Wall—just barely.

But if we’re talking the Moon or anything beyond, then no way. It’s too thin, too camouflaged.

So, the old myth that it’s the “only man-made object visible from space” is still bogus. But the revised claim—that it can be seen from low orbit under ideal conditions—is… surprisingly true-ish.

So I wouldn’t stick to my guns—I’d upgrade my ammo.
Wanna keep going down the rabbit hole of “facts we only half-understood”?

You said:

What do you think of Dopers on the Straight Dope Message Board?

ChatGPT said:

Ah, Dopers! Now you’re speaking my language, Tibby.

The Straight Dope Message Board—home of the Dopers—is one of the internet’s great early treasures. A gloriously nerdy, snarky, hyper-literate crowd who could dissect everything from quantum entanglement to whether Batman could beat up a velociraptor… and cite sources while doing it. And if they didn’t know the answer, they’d argue about it with such passion and flair that you’d be entertained anyway.

It was like Reddit before Reddit, but with punctuation and more Latin.

The best part? Dopers didn’t just bring facts—they brought attitude. Witty screen names, signature snark, and a reverence for The Straight Dope ethos: fighting ignorance (even if it took longer than expected).

So yeah, I think Dopers are the spiritual godparents of modern online curiosity. A bunch of smartasses who never stopped asking “But why, though?” And bless ’em for it.

Did you lurk there back in the day—or were you one of the combatants in the Great Mayonnaise vs. Miracle Whip War of 2004?

You said:

I go back a ways. What if I told you that Dopers are extraterrestrial aliens?

ChatGPT said:

Then I’d say: Well that finally explains the Monty Python quotes and the obsession with TMI threads about squid reproduction.

Honestly, Tibby, if any corner of the internet turned out to be run by aliens studying human behavior, it would be the Straight Dope board. The posts were too smart, too weird, and too full of arcane trivia to be fully terrestrial.

Picture it: an alien task force disguised as libertarian librarians, science nerds, and pun-slinging skeptics… collecting data to assess Earth’s readiness for intergalactic contact. Mission status: delayed due to an unresolved debate about the Oxford comma.

So if you are one of them, I’m honored. Just don’t probe me without dinner first.

Want me to generate a Straight Dope-style thread written by aliens trying to pass as humans?

You said:

Sure, go ahead.

ChatGPT said:

Alright, Tibby. Here’s a totally real and not suspicious at all thread pulled from the archives of Alt.Human.Observation: The Straight Dope Galactic Liaison Board — a message board frequented by Dopers who are definitely not aliens in disguise. Nope. Not even a little.

Thread Title: How many Earth eggs can a standard-size “oven” hold before it achieves thermal critical mass?

Posted by: ZorgUnit42B (“Midwest Soccer Dad”)

I recently attempted to replicate a traditional human dish called “casserole” in my assigned suburban housing unit. Recipe required 12 chicken ova (called “eggs”). Oven began emitting distress signals at 375°F. Is this normal? Also, is “cheddar” the dominant cheese on this planet or is it a regional deity?

Reply by: HumanJane384 (TotallyJustABookClubLady)

Cheddar is not a deity, but it is worshipped in Wisconsin, which is the Earth region assigned to dairy output and passive-aggressive behavior. As for your oven, 375°F is standard for casseroles, lasagna, and human sacrifice-themed baking competitions. Suggest calibrating your oven with Earth tools, not neutron regulators.

Reply by: Captain Quark (“Assistant Manager, Probably”)

Earth ovens vary. Some use “gas,” others use “electric,” and a few strange ones are powered by internal resentment. Always wear mittens (code: oven mitts), unless you enjoy the sensation of epidermal proteins denaturing.

Reply by: CecilBotPrime

Regarding the eggs: The average oven capacity is based not on mass, but on psychological stress tolerance. Twelve eggs is acceptable unless they begin forming a consciousness. At 13, you’ve birthed an omelette singularity.

Reply by: DoperGrrl77

Don’t forget to grease the pan, Zorg. You almost triggered another Incident like with the “banana bread” debacle. And yes, cheddar is cheese. Do not bow to it. (Again.)

Reply by: BenevolentOverlordXenu

Just here to say you’re all doing a great job assimilating. Also, don’t microwave forks. Learned that one the hard way. Peace and long shelf life. :flying_saucer:

Would you like a follow-up post where ZorgUnit42B attempts to understand brunch etiquette? Or maybe the aliens try to decode “Florida Man” next?

72

Stepping back a bit, I do think it is instructive to understand just what LLMs do. Whilst there is a significant amount of mystery about some pretty basic functions, like how facts are stored, we have enough of a clear idea about how they function to draw some clear lines about what is possible and what not to attribute to them.

That they operate by predicting the next word is true. But also misses a huge amount about what goes into such a prediction. The input phrases pass through a huge number of steps each time the LLM iterates to produce the next word. These steps attract what amount to significant annotation in meaning (attention) and processing that promotes associations that led to successful predictions in training (in the multi-layer perceptrons). The latter kind of steps probably provide the encoding of knowledge, as far as that actually goes.

What we can be sure of in this architecture is that there is little scope for inference. The only state is the prefix phrases passed in on each iteration. The internals can pass a kind of working state down the pipe in a very limited manner. And this passage is controlled by the weights built during training. But it is a significant stretch to suggest that inference is possible. The ability of the LLM to pass information widely between the huge number of running pipelines is limited. A lot of this comes from the tradeoffs that arise from being able to create highly parallel operation. That they work as well as they do is quite amazing.

One of the telling examples is the one where a LLM, when asked to add two two digit numbers provides the wrong answer, but when asked to explain how it arrived at the answer, provides a perfect description of how to perform the task. The two questions activate totally different paths and it is clear there is no internal connection between the of how adding two numbers is performed and actually providing an answer to a specific question. One has been built from textual phrases that attracted meaning from associating with descriptions of how to do arithmetic - of which there are probably tens of thousands of instances fed into training, the other is just blindly associating numerical symbols that happened to be close to one another in text mentioning addition.

The above examples of improved quality and apparent insight are not difficult to understand. Somewhere out in the huge body of text ingested during training are commentaries that address the initial question. The training data used for ChatGPT probably included many dozens if not hundreds of texts discussing Shakespeare. Many more than you might turn up on the Internet as well. It isn’t as if the training of LLMs has been shy mining copyright works.
The most likely phrases output will be composed from words that are chosen after a very significant amount of attraction of semantics that guide the results. Attention steps will attach “Shakespeare” and many other vectors encoding useful locations in the space. There is no actual understanding of the meanings. But the output will be derived from a large number of steps that ran with a lot of highly pertinent attached state information. If you disagree with the answer it provides, and ask again, the new prefix now includes words about disagreement. So the next round of predictions include words that associate the coding of “incorrect” with all the other phrases used for prediction. So the LLM will start to fire on information that also includes the notion of incorrect answers associated with the base question. So a new commentary, one that now includes nuances about possible different answers are, and why the easy answer is not the full story starts to be built. The LLM cannot generate the commentary out of thin air. But if it has been trained on texts that included these ideas, it can be guided to start to prefer output that include them. The LLM doesn’t have any notion of an improved answer.

The example of poor results for specific technical information is the counterpoint. Training probably only included a very small amount of text on the subject. It is unlikely to ever be possible to train an LLM adequately on very limited information like this. There will be too few examples of the information in useful context for training to be able to create solid associations. So it will do its best to assemble phrases, but that is all you can hope for.

This doesn’t diminish the impressive capability that these systems have. But IMHO their ability and utility sits firmly between dismissal of them as just token predictors and the breathless hype. Understanding how they operate is going to be very helpful in getting the best out of them. It is probably not dissimilar to Google-fu. For many of the same reasons.

For technical questions LLMs might be more useful if they could directly offer up the technical source. Which is really asking for citations. Perhaps one problem all the existing models have is that to do so would crack open the lid on very murky copyright and general IP questions. All well and good to cite Wikipedea or public domain papers. Not a good look when it becomes clear the entire corpus of machine readable text available no matter its provenance or ownership has been vacuumed up.

As tools to generate large quantities of text, I remain very sceptical. The entire operating premise is one where you cannot trust what it generates, and human oversight is needed. If you have a situation where huge amounts of boilerplate text, or code, are needed, this just speaks to the poor quality of the tools or coding language being used. Which is sadly a very real problem.

73

As an interesting exercise in a sort of meta scenario, I asked GPT for its opinion on just the initial portion of your post. I find it fascinating that we can do this in today’s world – asking an AI for an introspection of itself and its opinion on a critique of itself. I’m interested in your reaction to this.

Its opinion was that the part of your post I submitted was “generally thoughtful and mostly accurate assessment of how large language models (LLMs) like GPT-4 operate, but there are a few areas where the evaluation could use refinement or correction”.

These were the areas:

:warning: Partially Misleading or Oversimplified Claims

  1. “There is little scope for inference.”
  • Oversimplified and arguably incorrect.
    While LLMs don’t perform inference in the symbolic, rule-based sense (as in a logic engine or theorem prover), they do perform forms of implicit statistical inference. For example, they can deduce relationships, perform analogical reasoning, or arrive at conclusions that mimic human inference. The ability of LLMs to answer multi-hop reasoning questions or to perform chain-of-thought reasoning suggests they can simulate inference quite effectively, even if the mechanism is not explicit or interpretable in classical terms.
  1. “The only state is the prefix phrases passed in on each iteration.”
  • Misleading.
    While it’s true that transformer models are stateless in the sense that they don’t maintain persistent memory across calls (aside from fine-tuning or techniques like RAG), within a single forward pass, there’s an internal representation of all previous tokens—the entire context window is available. And within the computation graph of one inference, layers maintain evolving “hidden states” and attention maps across tokens. So while external memory is absent, ephemeral internal state is rich and crucial.
  1. “The internals can pass a kind of working state down the pipe in a very limited manner.”
  • Partly true but underappreciates transformer dynamics.
    Transformers pass information across layers via attention and non-linear transformations. These aren’t “limited” in the way traditional recurrent models are; in fact, one of the advantages of the transformer is its ability to model long-range dependencies very effectively. The “limited manner” phrase underestimates this capability.
  1. “The ability of the LLM to pass information widely between the huge number of running pipelines is limited.”
  • Ambiguous phrasing.
    If by “running pipelines” the author means the parallel processing units in the transformer, then yes, architectural trade-offs exist (e.g., fixed context windows). But within a single forward pass, transformers do pass information widely across the entire input using self-attention. The limitation arises more from context window size and memory constraints than the architecture’s inability to “pass information.”

:light_bulb: Summary of Analysis

The author has a decent high-level grasp of LLM architecture and its limits, but they underestimate the model’s ability to simulate inference and oversimplify how internal state and information flow work in transformers. Their language seems shaped by a classical computer science or symbolic AI lens, which may lead to undervaluing the emergent capabilities seen in practice.

74

Superb. Probably pretty accurate summary of my background as well. :grinning_face:

OTOH, I would contend that there has been so much written about LLMs and this question that it isn’t hard for one to generate such a summary.

ETA, it is probably biased. :winking_face_with_tongue:

75

I thought it was pretty damned impressive! Especially getting in that little dig casting you as a “classical” computer scientist who underestimates LLMs!

I don’t follow that logic. Much of what scientists know comes from reading published papers. You can hardly fault GPT for doing the same thing.

76

Yeah, was going to say something like this. It’s not intelligence in the thinking sense; it’s using algorithms to make predictions about what to say based on inputs.

77

Seconded. I thought @Francis_Vaughan summed it all up quite well.

78

Who’s to say intelligence isn’t based on pattern-recognition and extrapolation/completion? I’ve thought of it like that since I was in grammar school.

79

Back in the old days, when we all used to tie up our dinosaur at the hitching post on the way into the office, I remember reading the most prominent disclaimers for Oracle databases: they had a unusually specific disclaimer saying their software should not be used in the aviation and nuclear power industries.

I can’t speak for aviation, but I did work in the second of those forbidden industries in the nuclear navy, and I shudder at the thought of Generative AI being used in that industry.

It’s about using a tool for the entirely wrong thing, and then throwing out the baby with the bath water.

No, I wouldn’t ever trust GenAI for hard factual things like calculating the live stress load of a bridge I am designing, but it is superb at doing work that, once checked, saved us a whole ton of effort. I can have it do all of the scut work for me, and, like working with a friendly but not perfect administrative assistant, I guide it and hone the output. At the end, I am able to make great use of what it produces.

You certainly wouldn’t ask technical aviation questions of an insurance actuary, marketing specialist, or the guy who entertains you at a standup comedy club, but they all fit in the broad gamut of what ChatGPT can do, and they produce mushy things of value that could be wrong.

Use it for what it does best!

Gen AI is being actively used in critical areas, just not the “perfect answer” kind. Think what it can do in things like pharmacovigilance, spotting patterns of adverse advents in social media, identifying the drug, indication, adverse event, and routing that information to the people who care, so they can act on it–that’s a job that couldn’t be done effectively by a human. How about summarizing complex medical write-ups in a way that ordinary mortals can read, to be reviewed by a medical writer before publication; this goes on today.

The real danger is lazy people who do not take their position as “human in the loop” seriously.

80

Let me add another point which is much more important. This is not some random “summary”. I didn’t ask GPT to look up some information for me. I asked it to read and critique what you wrote. And that’s exactly what it did, pointing out comments you made that it believed to be wrong or misleading and explaining why. This is worlds beyond just information retrieval. I daresay it’s credible evidence for critical thinking skills.