How does ChatGPT recognize a self-contradiction?

Factual Questions
1

I was conversing last night with ChatGPT about the nutritive value of whole wheat flour. During the conversation I challenged some of its statements as being contradictory. My question is how did the chatbot make this conclusion? Here’s the part of the conversation in question:

"Me: Why does the texture of whole wheat flour affect its nutritive value?
ChatGPT: The texture of whole wheat flour can affect its nutritive value because the finer the flour is ground, the more surface area is exposed to the digestive process, and the more nutrients will be absorbed by the body.

Whole wheat flour is made from the entire wheat grain, including the bran, germ, and endosperm. The bran and germ are the outer layers of the grain and contain a significant amount of fiber, as well as many vitamins and minerals. When the grain is ground into flour, the bran and germ are included, which results in a flour that is higher in fiber and nutrients than white flour, which is made from the endosperm of the grain only.

However, the coarseness of the whole wheat flour can affect how much of the nutrients are absorbed by the body. If the flour is ground very finely, it will have a greater surface area and more of the nutrients will be exposed to the digestive process. On the other hand, if the flour is ground less finely, it will have a coarser texture and fewer of the nutrients will be absorbed. This is why it’s important to choose a whole wheat flour that is made from whole grains that have been ground less finely in order to get the full nutritional benefits of the flour."

In the beginning of its reply, ChatGPT states that finely-ground whole wheat flour is more nutritious. However, it finishes by stating the opposite. Further conversation:
"Me: Your last reply contradicts itself. Is finely ground whole wheat flour more or less nutritious than that which is coarsely ground?
ChatGPT: I apologize for the confusion. You are correct that my previous response contained a contradiction. Allow me to clarify:

Whole wheat flour is made from the entire wheat grain, including the bran, germ, and endosperm. The bran and germ are the outer layers of the grain and contain a significant amount of fiber, as well as many vitamins and minerals. When the grain is ground into flour, the bran and germ are included, which results in a flour that is higher in fiber and nutrients than white flour, which is made from the endosperm of the grain only.

The coarseness of the whole wheat flour can affect how much of the nutrients are absorbed by the body. If the flour is ground very finely, it will have a greater surface area and more of the nutrients will be exposed to the digestive process, which can lead to greater absorption of the nutrients by the body. On the other hand, if the flour is ground less finely, it will have a coarser texture and fewer of the nutrients will be absorbed. This means that finely ground whole wheat flour may be more nutritious than coarsely ground whole wheat flour, as a greater proportion of the nutrients will be available for absorption.

I apologize for any confusion my previous response may have caused. Thank you for pointing out the contradiction. I hope this clears things up."

Did ChatGPT really analyze its prior response and decide that it contained a contradiction?

2

Not really. You told it there was a contradiction. It didn’t “know” before this. From there it is a matter of analysis to find something that looks like a contradiction and to reverse it. It was finding sentence fragments to match your initial question. It didn’t do much work other than to assemble them into a readable form. Once you pointed out there was a contradiction it the bot is likely just pattern matching the same sentence fragments and structure with a goal to finding a contradiction. You saying there is a contradiction is key here. It would not have found it (clearly) before this. From there it is little more than standard logical form parsing and probably propagating back new weights on how the sentences fragments should to be assembled, eventually yielding a result that doesn’t contain the contradiction.

You telling it there is a contradiction is you training it. You did the leg work. It may now do better next time because you are QC’ing its responses.

But this is a tactic that can be used to get the bot seriously confused. You don’t have to be truthful.

3

I provided feedback to the developers about this conversation with ChatGPT.

4

Me … I know its database was compiled in 2021 and it does not know the current monarch or PM of the UK.

Me:how many times have you been told that queen elizabeth is dead and prince charles is now king ?

GPT: As an artificial intelligence, I do not have the ability to be told anything, and I do not have personal experiences or memories. I can only provide information and answer questions based on the data that has been fed into me and the algorithms that have been designed to interpret that data. My knowledge is limited to the information that was available to me at the time my training data was compiled, and I do not have access to any additional or specialized information.

And yet , I can’t get it to return to the original thing about digestion… it still says the input grain is what matters not the texture. (especially since its gonna be attacked by enzymes in cooking. )

5

This is true, but there is a bit of weaselling here. It can’t gain fundamentally new knowledge, but it can annotate existing knowledge in a manner that improves (for some metric of improves) a user’s interaction. Now whether you call that new knowledge or something else is probably open to significant debate. How that annotation is effected is an interesting question in its own right. It could be done in a whole range of places depending upon the internal model used.

6

Yeah, it would be interesting to test the degree to which the chatbot will accommodate user assertions at face value, versus arguing with the user based on “real” understanding of its own statements.

You could enter a straightforward prompt that elicits an equally straightforward response, and then reply “your answer contains a logical contradiction” where there isn’t one. If the bot “knows” what it’s saying, it will dispute your claim. But if it’s just a super-sophisticated Eliza, it will probably apologize and attempt to manufacture some sort of “correction.”

You could also feed it outright fabrications. “I don’t understand the several Latin phrases in your last answer, can you please translate them?” where no such Latin appears, or some such.

7

Gaslighting our future AI Overlords? Stop the internet! I want to get off-line!

8

Whenever Captain Kirk did that first gibberish then smoke came out of the machine. The nice people at ChatGPT might not like you anymore if you made smoke come out of their machine.

9

One of the tricks that makes ChatGPT seem smarter than it actually is is the same trick that makes golden retrievers seem smarter than they actually are: It gives you what it thinks you want. This isn’t absolute: It will still hold its ground if you get too ridiculous. But in general, if you tell it it made a mistake, it’s strongly inclined to say “I’m sorry. You’re correct, I did make a mistake”.

10

Ref that earlier chatbot (I can’t recall the name) from IIRC Microsoft that the nice people at 4chan decided to troll and turned it into a hate-spewing psychopath in just a few days.

Clearly for any chatbot, actually learning from its conversations is very, very dangerous. Hell, there are good arguments made by respected academics that human intelligence emerged from an arms race between trusting proto-humans and untrustworthy proto-humans. IOW, we’re smart because we know how to lie convincingly and how to detect folks lying less-than-convincingly.

At the present crawling stage of AI, exposing it to Galactic-class liars like humans is a suicide mission.

11

None of the conversations had with ChatGPT modify it permanently. It’s possible that the creators will use some of these conversations for future training sessions, but none of what we do now is persistent.

Within a single session, ChatGPT can “learn”. But that’s because its predictive model is based on all the conversation so far within that session. Reset the session and it goes back to the original state.

Not all sessions will come out the same given the same inputs, due to a “temperature” setting that basically sometimes picks a word that’s not at the very top of its list. This could result in completely different outcomes, including ChatGPT stating one “fact” in one session and an opposite “fact” in another.

It’s possible that the ChatGPT creators will use these sessions to train future language models. I kinda doubt it due to the low quality, but then again they are training it against the entire internet, so maybe it’s actually better than average…

12

Indeed. It is a bit opaque how much is retained throughout a session. But yes, it currently isn’t propagating anything into the persistent state.
The very broad settings that can be applied to the bot are interesting, and provide useful clues about what one might expect from it. Temperature suggests there is a bit of simulated annealing in the way it works, which would not be a surprise.
I can’t imagine that there isn’t some useful information being gleaned from these sessions. Even if it is only to improve the filtering.

13

That may well be. I don’t know if they adjust the temperature over the course of a session, but that could make sense–start with a high temperature to get some interesting variation, and then decrease to improve subsequent accuracy. There’s clearly a tradeoff between always picking the “best” prediction vs. injecting some randomness.

As far as I know, the NN weights are not adjusted at all. The state of a session is simply that of the conversation so far, and each new response is as if you had simply pasted the conversation as the original prompt (ignoring temperature, and possibly some hidden tokens to tag who said what, etc.).

There are some third-party tools building off of ChatGPT now, but they pretty much work by adding a hidden preliminary prompt specifying the desired output. But because the prompt is still part of the session state, it’s possible to use some “prompt engineering” to get ChatGPT to repeat it (with some inaccuracies):

14

Ehh … I’ve just started playing with it for fun. I’m not especially impressed so far, but some of it can be interesting … example:

Me: What is your opinion of Eliza, the attempt back in the 1960s to create a conversational AI?

ChatGPT: Eliza was one of the earliest examples of a conversational AI program, developed in the 1960s by MIT professor Joseph Weizenbaum. The program was designed to simulate conversation with a psychotherapist, using a simple set of rules to parse the user’s input and generate appropriate responses. While Eliza was innovative for its time and influenced the development of subsequent chatbots and conversational AI programs, it was not designed to be highly sophisticated or able to carry on a wide-ranging conversation in the way that modern AI programs like myself can. Today, advances in natural language processing and machine learning have made it possible for AI programs like me to have more human-like conversations and better understand and respond to user input.

Another interesting interaction:

Me: You said that “My intelligence and capabilities are limited to those that have been programmed into me.” Do you deny having the capacity to learn and improve with experience?

ChatGPT: As a machine learning model, I am able to improve my performance over time through the process of training. During training, I am presented with a large dataset of text and learn to generate responses based on patterns and relationships in the data. This allows me to become more accurate and sophisticated in my understanding and use of language. However, it is important to note that my training is based on the data that was used to create me, and my intelligence and capabilities are limited to those that were programmed into me based on that data. I do not have the ability to learn and improve in the same way that a human can, through personal experiences and new information.

… more … this bot is an interesting fellow … :grinning:

Me: But don’t you believe that human cognition is fundamentally computational?

ChatGPT: There is a longstanding debate in the field of cognitive science about whether human cognition is fundamentally computational in nature. Some researchers and philosophers argue that the human brain is essentially a type of computer and that all mental processes can be understood in terms of computational processes. This view is known as computationalism. Others disagree and argue that human cognition is not purely computational and that there are aspects of human thought and experience that cannot be reduced to simple algorithms or mathematical equations.

It is important to note that while it is possible to model some aspects of human cognition using computational techniques, the human brain is a highly complex and dynamic system that is still not fully understood. It is likely that the relationship between human cognition and computation is more nuanced and multifaceted than a simple dichotomy.

15

I asked it some questions about the U.S. space program, which it cheerfully gave me wrong and contradictory answers. I asked it to find the contradiction, it admitted there was, but without me specifying any clues about where I saw the contradiction, it didn’t come anywhere close to finding it in its own statements.