Why ChatGPT doesn't understand

I confess I lean in that direction, in as much as I consider the physical matter of a human brain to be all that there is and as such it is ultimately amenable to human replication or approximation.

It is merely a question of scale and incremental increases in our understanding of how it does what it does.

I appreciate that “merely” is performing some hurculean heavy lifting there.

'Zactly. You’ve captured the zeitgeist perfectly.

And yes, pretty much nobody who knows what they’re talking about thinks the current generative LLMs like CHatGPT can “think” in the everyday general sense.

But progress is going that way at a good clip. What’s unknowable just yet is how far the journey is. So even fast incremental progress may still take a very long time to get to the distant destination. Or maybe we’re darn near there already. Nobody has a map to the territory ahead.

“Nautical Disaster.”

One thing I’ve found ChatGPT to struggle a bit with is when I tell it to insert punctuation (such as commas, capitalization) into a block of text but not change the words themselves. The AI often cannot resist re-wording and re-phrasing things to make it “better,” deleting or altering entire sentences.

I don’t think that’s the case - or, at least, I can’t find a source saying it is.

ChatGPT is already hard-prevented from a lot of outputs around violence and bigotry.

Am I the only one not that impressed with ChatGPT? The only amazing thing about it is that it generally produces coherent English. But if you try to understand the meaning of what it says, it is often nonsense.

No you’re certainly not. I personally remain impressed by what it can do and use it in ny life professionally.

Well, new yorkers are rather outré.

I guess it depends what you want to use it for. For creative writing exercises where the validity of the output isn’t important, I guess it’s fine. I’m looking for a device that, when I ask it to solve a problem or to give me some factual information, it gives me a reliable answer. So far it fails a lot of the time.

TBH, what I really want is something that can help me solve British cryptic crossword puzzles. Nothing helpful remotely exists right now, and that include ChatGPT.

Yes, it’s not going to work for that. It does help me with stuff like tip-of-the tongue syndrome. If I’m looking for a word and can’t quite find it mentally, giving it a description of what I’m going for works more often than not. The other day I couldn’t for the life remember a certain Photoshop shortcut, so I asked it “what’s the shortcut for when you want to merge all visible layers as a new, single layer?” and it informed me it was CNTL-Alt-Shift-E, the correct answer. I also couldn’t figure out something in my car’s infotainment system, and it even managed to get the right for that, as well. There are certain tasks it’s great at, others where it just makes up shit. I have a pretty good feel on what it’s good at, so I use it for that. Typically, I use it most to helping me with emails, snail mail, and just in general thinking things through and coming up with plans. It’s been a few months, and I still think this technology is fucking incredible.

The main thread on this subject has lots of examples of ChatGPT solving problems and giving factual information. I see you’ve participated in that thread but it’s very long and you may have missed some of these examples. Granted that it has no provision for fact-checking or confidence rating so its responses can, indeed, sometimes be wrong, but even in its current form it’s a very useful general tool for comprehensively answering questions, provided you’re willing to do some quick sanity checks. I’ve long since stopped regarding it as a mere conversational toy, and I and many others actually find it genuinely useful for information retrieval because of its staggeringly vast collected body of knowledge. I think you’re vastly underestimating a major technological breakthrough with almost unimaginable potential.

If you have to fact-check the result it gives anyway, what’s the point?

At this point I believe it is important to specify if you are talking about ChatGPT v 3.5 or v4. They are vastly different in capability. I have found ChatGPT 4 to be well worth the $20/month price.

I’ve only been using 3.5, though I have 4 available through Bing, but I don’t like it as much for some reason. There’s something about the way Microsoft has it set up that irks me. I should try the OpenAI paid version. I like their interface best.

A couple of comments:

First, most of the ‘this is nothing great’ commentary seems to be about ChatGPT 3.5, which is now one of the weaker models around. Try Bing Chat in creative mode or GPT-4.

Second, there is an art and science to prompting these things, and I suspect a lot of people getting disappointing results have not learned how to do effective prompts. The size of the input context is anywhere from 4K to 32K depending on the model, yet people expect good results from typing in a sentence. Learn about few-shot prompting, fine tuning and other techniques you can use to get better results.

One last thing: We are complaining about the writing capability of generic language models. A model developed specifically for creative writing will likely be fine-tuned. For example, we have yet to see how an AI would fare writing an episode of The Simpsons after having been fine tuned on every Simpsons script ever written.

Because if you’re looking for something obscure, Google might not have the first clue what you’re looking for. But ChatGPT does. And once ChatGPT lays out a comprehensive response, you can spot-check it to see if by chance the wording has sent it down the wrong path or whether, as it usually does, it hangs together factually. If so, you may have saved hours of time and even received information not publicly available anywhere else, or at least, not prioritized by Google.

All systems sometimes make mistakes, including human ones. IBM’s Watson system specifically tailored as a medical advisor has been known to give bad advice, and so have human doctors. Access to a very large body of knowledge is still extremely useful provided its limitations are understood.

MS shot GPT4 through the knees for their Bing version. The OpenAI interface is def the way to go.

It would probably do quite a good job in terms of writing something that was distinctly recognizable as containing all the familiar Simpsons tropes. On one hand, this is the basis of the fear that unscrupulous studios will exploit AI to generate rote scripts without employing writers. OTOH, I imagine that most folks watching TV series anticipate creative and imaginative surprises like novel situations and new characters that are unlike anything that came before but that nevertheless embody the spirit of the show and its main characters. I think it would still be a challenge for any current AI to bring that kind of creativity to bear.

I guess I’ve just never yet asked it a question for which I got the correct answer, and which I couldn’t have got the correct answer just as fast on Google, or even Wikipedia.

Well, I’ve had some long and involved conversations with ChatGPT that prove otherwise. Very simple examples are hard to come by because these other “dumb” systems can usually handle the simple ones quite well. But here’s one simple example that illustrates the point.

In AI systems, a “query agent” is a generic term sometimes used to describe the software layer that mediates user queries, acting as an intermediary between the user and one or more databases. Suppose I wanted to find out more about such systems.

If I type “query agent” into Google, I get a list of literary agents, with helpful information for aspiring authors on how to find an agent! :roll_eyes:

If I type “query agent in artificial intelligence”, I get a hodgepodge of articles, some of which are irrelevant, some highly technical, and none of which really explains the concept or offers alternative terminology.

If I type “query agent” into Wikipedia, I am told there is no such article, but it does helpfully ask me if maybe I meant “queen agent”!

If I ask ChatGPT what a query agent is, it infers I meant in the context of AI, and it provides a four-paragraph summation. I can then carry the conversation further into an area of interest, by asking it, say, if IBM Watson has such an agent. It replies that yes it does, and describes the IBM Watson Discovery service and its query agent. I then ask it (out of genuine curiosity) how that’s different from IBM Watson Assistant. It describes the differences in detail, essentially amounting to the fact that Watson Discovery is a query and information retrieval system, while Watson Assistant is a natural language conversational interface in the manner of a chatbot. Intrigued, I ask it whether Watson Assistant and Watson Discovery can work together, and ChatGPT confirms that they can, and provides a series of steps illustrating how this will typically be done.

This shows how, beginning with an intelligent understanding of the very first question (as opposed to Wikipedia’s “did you mean ‘queen agent’?” :rofl:) ChatGPT can be guided to provide more and more detailed expository responses precisely tailored to the matter of interest.