# Why ChatGPT doesn't understand **URL:** \ **Category:** Great Debates\ **Tags:** ai\ **Created:** [February 26, 2023, 6:13pm UTC](https://boards.straightdope.com/t/why-chatgpt-doesnt-understand/980404 "2023-02-26T18:13:29Z")\ **Posts on this page:** 5\ **Page:** 10
**Author:** ![Sam\_Stone](https://avatars.discourse-cdn.com/v4/letter/s/ecccb3/32.png) [@Sam\_Stone](https://boards.straightdope.com/u/Sam_Stone)\ **Post date:** [May 22, 2023, 9:04pm UTC](https://boards.straightdope.com/t/why-chatgpt-doesnt-understand/980404/182 "2023-05-22T21:04:13Z")
Try using Bard, or bing chat. Both have connections to the internet, can look up information, and will cite their sources. ChatGPT with plugins will do that and much more. For example, you could tell it, “Find me a vacation deal for someplace warm in January, but I don’t want to connect through Toronto Pearson because it’s a hell hole” and it will use Kayak, Travelocity and other services to find what you want. Doing that search yourself would be tedious as hell. There are a lot of queries like that. Queries that require sub-queries, multiple queries, etc. Here’s one I did a while ago: Try to find the city in Canada which was the largest coal mining city. Bing got it right away. I then tried to do it the ‘traditiinal’ way, and it led me through a whole bunch of web sites on the history of Canadian coal mining, none of which really answered the question. ---
**Author:** ![Mr.E](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mr.e/32/8932_2.png) [@Mr.E](https://boards.straightdope.com/u/Mr.E)\ **Post date:** [May 22, 2023, 9:12pm UTC](https://boards.straightdope.com/t/why-chatgpt-doesnt-understand/980404/183 "2023-05-22T21:12:12Z")
> [@Mnemnosyne](#): > > If I am not mistaken, that’s in part because this model is specifically prohibited from learning. It cannot actually learn from input not because that can’t be done, but because it’s been restricted from doing that so that the internet can’t teach it to be an offensive nazi racist douchenozzle, because that’s what happened to other, more primitive chat models when they were allowed to learn from public input. This is partially true, but more broadly the method of training the current slate of models is intensely expensive and computation-heavy, much moreso than responding to prompts is. There simply isn’t enough time or processors to continue training a model like this in real time on user prompts. That said, they are absolutely storing user prompts and model responses and using that to train further generations. ---
**Author:** ![Sam\_Stone](https://avatars.discourse-cdn.com/v4/letter/s/ecccb3/32.png) [@Sam\_Stone](https://boards.straightdope.com/u/Sam_Stone)\ **Post date:** [May 22, 2023, 9:24pm UTC](https://boards.straightdope.com/t/why-chatgpt-doesnt-understand/980404/184 "2023-05-22T21:24:15Z")
Well, the context is a form of ‘training’. It’s just not very large and only lasts for the session, then is reset. When models are ‘fine tuned’, it’s a similar process, but permanent. For example, you could fine-tune a model on all of your corporate operations manuals, then use it for checking ISO 9001 compliance. The ‘training’ that is expensive for LLMs is the initial huge dataset that makes them them able to do general tasks, understand natural language, etc. From there, ‘fine tuning’ isn’t very expensive. And there’s an open spurce LLM out there now that is ChatGPT 3 level, and was fully trained for $30,000, on completely open source data sets. And Alpaca, an LLM that can run on a laptop, was trained with a different technique: they had GPT-4 train it. The result is a model with only 12B parameters yet functions almost as well as GPT 3.5. ---
**Author:** ![Mr.E](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mr.e/32/8932_2.png) [@Mr.E](https://boards.straightdope.com/u/Mr.E)\ **Post date:** [May 22, 2023, 10:03pm UTC](https://boards.straightdope.com/t/why-chatgpt-doesnt-understand/980404/185 "2023-05-22T22:03:42Z")
True, the fine-tuning training is not as expensive as the initial training. It’s usually a trade-off, you make the model more specialized to the output you want but you’ll start to get responses that always tend toward the fine-tuning. Like, I’ve seen Stable Diffusion models trained to produce a very specific kind of painterly Anime art but they struggle to do anything other than that kind of image. The “context” you get in the chat-like environment of ChatGPT is even simpler, all the previous interactions you have in that chat are fed back into GPT with every prompt, up to whatever the current token limit is. You can tell ChatGPT something and it will “remember” it only because it gets re-told everything again each time you submit a prompt. ---
**Author:** ![I\_Love\_Me\_Vol.I](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/i_love_me_vol.i/32/521_2.png) [@I\_Love\_Me\_Vol.I](https://boards.straightdope.com/u/I_Love_Me_Vol.I)\ **Post date:** [May 26, 2023, 12:04pm UTC](https://boards.straightdope.com/t/why-chatgpt-doesnt-understand/980404/186 "2023-05-26T12:04:07Z")
> [@Mr.E](#): > > You can tell ChatGPT something and it will “remember” it only because it gets re-told everything again each time you submit a prompt. Just like granny! [Previous page](https://boards.straightdope.com/t/why-chatgpt-doesnt-understand/980404.md?page=9)