I can take a shot at explaining it.
Don’t think of AI as behaving like a human using Google—where you input a query, scan multiple results, and make a thoughtful, discriminating choice.
Large Language Models (LLMs) operate differently. They’re probabilistic models trained on massive data sets. At each step, they generate output by selecting the most statistically probable next word, given the preceding context. This process continues recursively, word by word, until the response is complete.
Because the model can only reflect its training data, it doesn’t “know” facts—it estimates what a plausible answer might look like. And since the training data includes both high-quality and low-quality sources, the output can reflect either.
One example I found intuitive was in a recent OpenAI paper: when asked about the Māori language, a smaller model might respond with “I don’t know.” A larger model, having seen some Māori data, might try to answer, ‘thinking’ it has a viable probability path.
People are talking about creating checks on hallucinations, mid-process, and OpenAI says they’re doable, but nobody has implemented them yet (that I know of).
Ninja’d! (also, we have a thread discussing some of this stuff)