Why ChatGPT doesn't understand

V4 also got the silver question correct:

Are there any any alloys of silver which have better electrical conductivity than pure silver?

ChatGPT

No, there are no alloys of silver that have better electrical conductivity than pure silver. In fact, silver is the most electrically conductive element, with a conductivity of approximately 6.3×10^7 S/m (Siemens per meter) at room temperature. When other elements are added to form an alloy, the electrical conductivity of the resulting material generally decreases due to the presence of impurities and disruptions in the lattice structure.

At the rate we are going, when someone says a chatbot can’t do X, they should qualify it with “At least, for the next day or so…”

These things are improving at an accelerating rate as they get reinforcement feedback from billions of conversations and constant hardware and model improvements. Also, I believe the addition of multimedia to the GPT-4 training data made its text answers better - especially in the sciences and math.

Yes, I did ask it exactly that. It is a well known ‘trick’ question, which any human paying attention will see through. I wanted to see if the bot would do so.

Call me a Luddite, but I’m not at all happy with the direction these developments are taking. There is enough rubbish out there on the Internet already without the addition of bots which make up a completely bogus answer (and don’t tell you they are doing so).
A mark of an intelligent and educated person, IMHO, is that they are not afraid to say “I don’t know” when they don’t.
But you can’t put the genie back into the bottle, and any attempt to build Asimov-type laws into these things is doomed to failure, if only because someone else will just create a version without the restrictions for nefarious purposes.
We will just have to keep sharpening our bullshit detectors, I guess.

Aah. Thank you. Not a trick question I had seen before.

And in fact I can come up with a reasonable alternative parsing of the question that gives a different answer. Clearly my parsing is not the one the question poser intended, but English is like that, full of multiple plausible ways to decode sentences.

More that due to the alignment problem Asimov’s laws are complete nonsense, there’s absolutely no way for us to tell a robot “follow these laws in this order” when the laws in question are nebulously defined instructions in English like the 3 laws.

Right, I did say ‘if only because’. But I agree that it would be difficult to build Asimov-type laws into a software system, the paradigms are so different. Of course Asimov was writing at a time when very few people (Turing, perhaps?) had any clear idea of how a real world computer might actually work.

In the eternal race between offence and defence, I am wondering if we will see the rise of ‘bot detectors’.
By analogy with adblockers? Trained on large samples of bot-generated text, perhaps.

There are already some plagiarism checkers which claim to flag stolen material in student papers etc, though I don’t know how effective they are.

You’re watching a stage play - a banquet is in progress. The guests are enjoying an appetizer of raw oysters. The entrée consists of boiled dog.

Um, this is so oblique from the original topic that I don’t think I see what you are getting at?
In a stage play, the actors will be pretending to eat ‘prop food’ which is probably just bread or jello?

Dog is better roasted though, at least as I understand it is eaten in some easten cultures.

It was a reference to Blade Runner - one of the Voight-Kampff test phrases in which the response is used to detect replicants. :wink:

Ah, I get it. As it happens, I have never seen that movie, perhaps it is worth watching.
I think we will need more sophisticated test phrases to detect bot activity in the future though…

Yes, worth your time IMHO. One of my favorites. The actor Rutger Hauer has a shining role.

A few weeks ago for some joke reason I asked ChatGPT to translate a song’s lyrics to Latin. It produced a bunch of Latin, a language I don’t understand, but headed the stanzas with things like “Chorus,” which was funny because the song in question doesn’t have a chorus.

So I backtranslated and it became apparent ChatGPT had pulled up the first two thirds of the song’s lyrics correctly but the last third were a totally different song. Well, ok. I told it that and asked it to recheck. “Sorry, I got it wrong, here’s the real lyrics…”

… And it did it again.

And again,

And again,

And again.

It COULD NOT learn the lyrics of an immensely well known song, lyrics reposted all over the internet, even when told precisely where the error started. It was like trying to teach a sofa.

Now that is some great imagery! Thank you!

OTOH, I have learned some important life lessons on a sofa. And not from a nearby TV.

Incidentally, I tried again just now, same problem. Even if I tell it, in several different ways, precisely what errors it’s making, it can’t learn from them.

If I am not mistaken, that’s in part because this model is specifically prohibited from learning. It cannot actually learn from input not because that can’t be done, but because it’s been restricted from doing that so that the internet can’t teach it to be an offensive nazi racist douchenozzle, because that’s what happened to other, more primitive chat models when they were allowed to learn from public input.

If you had a similar model you could give input to which it was allowed to learn from, it probably would be able to learn from you telling it exactly what errors it’s making.

The next hard nut to crack, which they’re probably working on, is figuring out how to allow it to learn but discriminate between good input and bad input. This remains a difficult problem for humans, but on the upside, these AI models can be examined in much more depth, experimented on in ways you can’t do with humans, and the process can be iterated in reasonable spans of time, instead of decades.

What was the song out of curiosity?

An interesting discussion, plenty that is way above my head in areas that I rarely ever tread, but, if you’ll allow a layperson’s query.

To the best of our knowledge the human brain is a purely physical construct that receives and stores/connects/processes information and by doing so it enables us to interact with the world.

By our assessment of its interaction with the world we seem confident enough to say that a human, giving correct answers provided by that human brain, have an understanding of a given subject.

Is there any reason why a purely physical AI “brain”, performing equivalent data collection and processing, giving equivalent coherent answers, cannot be said to be understanding in an equivalent way?

That is perhaps a separate argument from asking if ChatGPT it is doing that now. If it is too much of a hijack feel free to ignore.

That’s one of the deep questions of philosophy. Most computer science folks think the answer is unequivocally & trivially that computer thinking and human thinking would be equivalent. Which is not to say they’d be the same.

But defining “thinking” as what humans uniquely do is unscientific.