Does consciousness require solving the halting problem, generating truly random sequences, or determining whether a Diophantine equation has a solution over the integers? If so, why?
I’m pretty sure dogs are conscious, and they can’t even conceive of those things, let alone solve them. Now, I kmow you just picked these as random examples, but how about a real example of something that can only be done by a conscious entity?
I don’t know exactly what consciousness is, but as a first assumption a good guess is that it’s just another feature that emerges in neural nets when they reach a certain scale and pevel of training. After all, that’s what happened with all the other capabilities these things developed.
A better approach would be to ask, “Why do we need consciousness? What does it bring to the table that system-1 type capabilities can’t do?” Forget the metaphysics, and focus on evolutionary needs and capabilties.
Why did conscious animals out-compete non-conscious ones?
What would you say is the most sophisticated animal that ISN’T conscious?
What differences are there between a biological brain and a digital neural net that might be required for consciousness?
We should try to identify which types of thinking might require a consciousness, then test the LLM’s capability to do that kind of thinking. And if there IS no type of thought that requires consciousness, why do we have it?
Our world is complex systems all the down. As complexity evolves, higher levels of organization emerge. It seems intrinsic to complex systems. You can see that in everything from galaxy formation to neural nets. As networks grow, they form clusters, superclusters, etc. It should be no surprise that if we make neural nets larger and train them on larger datasets there would be additional emergent structures. And that’s exactly what we’ve seen so far. Why should consciousness be different?
Well, the simple way to tell whether someone understands language is just to chat with them and see if that chat makes sense. ChatGPT passes that test. If one is to insist that ChatGPT doesn’t actually understand language, then one has to drag in all that metaphysics.
To the best of our knowledge, humans can’t do any of those things, either. If we could, then it would be definite proof that artificial humanlike intelligence is impossible. But we can’t.
I’m willing to let this argument die with the conclusion that we agree to disagree. I’m not trying to score internet points. 10,000 internet points and $3.65 will buy me a Caffe Latte Grande at Starbucks. But as long as you continue to make arguments that seem to misunderstand or misinterpret my position, I’ll continue to respond, at least until I get tired of it.
That’s true, and your remaining points are fair enough. But the statement that you pounced on was just a throwaway line at the end of what I thought was a valid point that this entire discussion is strikingly analogous to the Chinese Room argument. Simply put, Searle was trying to make the point that you can have a black box that appears to have a perfect understanding of Chinese, but look under the covers, and you find that there’s no understanding there at all. This argument has been widely despised in the AI community. I say stop looking under the covers; just test, observe, and draw your conclusions, because behaviour is all that matters.
Not at all. The concept of “understanding” is not a mystery. Per Wikipedia: “Understanding implies abilities and dispositions with respect to an object of knowledge that are sufficient to support intelligent behavior”. Understanding requires intelligence, so the tests for understanding are just a domain-specific subset of the tests for intelligence. The Turing “imitation game” asks whether an evaluator can distinguish which of the two participants is a machine by assessing its responses to a variety of questions and commentary. The test for understanding asks whether an evaluator can distinguish which of the two participants is a machine by assessing its responses to specific questions and commentary pertaining to a set of facts previously presented.
I’m not any sort of mathematician, but I think you’re referring here to Kolmogorov randomness (Kolmogorov complexity) which is a useful concept in information theory, but it’s not the only definition of randomness and doesn’t really seem relevant here at all. As you just said yourself, “For any arbitrary length, I can give you a computer program that will output a random number of that length that passes all tests for randomness”.
Why? That makes no sense. This feels like an attempt to conflate the loaded meanings of human-assigned names with the arbitrary labels that we might use to distinguish objects.
Let’s say instead that we have three objects, which we have arbitrarily assigned the labels Object_001, Object_002, and Object_003, so that we don’t confuse ourselves with overloaded terms. We then have the relations:
RNameAjax = {<Object_001>}
RNameBetty = {<Object_002>}
RNameChad = {<Object_003>}
RIsCat = {<Object_001>, <Object_002>}
Apply your push-through and you find that you can arbitrarily permute the three objects. And that’s perfectly fine since those were just arbitrary labels that we assigned to distinguish them. All that matters is that they’re different; they convey no other information. Ajax stays Ajax and Chad doesn’t become a cat.
What if we add some elements to our collection, like Ajax the dog? Then, although it’s still useless to ask what the label is, we can still distinguish the elements by stating things like “that object with a name of Ajax and is a dog” vs. “that object with a name of Ajax and is a cat” (name and type being the only relationships I’ve used so far).
Real relationships are obviously more complicated. And things like “name” and “cat” are themselves sets of relationships. Regardless, the point is that the underlying labels are arbitrary. They can be permuted at will as they have no underlying meaning.
For a given sentence, tokens so far assembled are presented to the net:
“It then takes the last part of this array and generates from it an array of about 50,000 values that turn into probabilities for different possible next tokens”
It doesn’t relate to any context or association to structure or even the numbers of the tokens involved. It’s an array of 50,000 lists of probabilities. It rummages through the array and picks one. It has no access to any contextual meaning associated with it’s selection. It returns the vector for its selection which allows the chosen token to be appended to the sentence and the process is repeated and the output routine gets another 50,000 lists to evaluate.
There is no mechanism in the machine to support understanding.
I believe using the term Artificial Intelligence is a huge social error. Tom Watson was adamant in his opposition to labels that implied a computer could think. Automated Word Processor. would suffice and not be misleading.
I think it’s a huge error to use the term “meat intelligence.” There’s no evidence that meat can ever have a true understanding of the tokens presented to it through its meat sensors, or that the tokens produced by squirting air through its flapping meat parts have any semantic relevance at all. It’s just ludicrous to pretend that meat can think.
Well, I give one in the linked article. Put briefly, in order to decide whether to self-modify to reach a certain goal, a mental state has to be certain that the modified version would still be capable of reaching that goal. In particular, it would need to convince itself that whatever the modified version concludes about the world, actually holds. That can’t be done computationally, due to Löb’s theorem. What I propose instead is that the self-referential nature of the process enables a mental state to introspect its own properties, beyond what a mere theory of itself can reveal to itself.
That process is not itself a conscious one, but rather, the basis of conscious experience. What can then be shown is that such a kind of mental state exists at a so-called modal fixed point: it’s equivalent to a certain kind of self-referential formula, that is in turn equivalent to a formula from which the self-reference has been eliminated; what remains is a belief about the state of the world as pertains to the goal it aims to achieve. Subjective experience (“qualia”) comes about via the non-theoretical, direct access to its own intrinsic properties.
Consciousness, in that model, then has the job of providing the substance of our beliefs (and other propositional attitudes) toward the world—of giving them a definite content. That is, its role is exactly to circumvent the Newman-type worries raised above: without it, our knowledge of the external world would be radically indeterminate.
Again, that begs the question at issue: whether understanding is a purely behavioral notion. That this very likely isn’t the case was one major reason for the downfall of behaviorism.
Sure. But the point was to answer @Sam_Stone’s contention that due to the emergent properties of LLMs, we can’t really say anything about their limits a priori. But we can say lots of things.
I didn’t ‘pounce’ on anything so much as merely point out that Fodor agreed with Searle on the basic setup of the Chinese room, which you seemed to imply he didn’t (‘Searle’s argument is particularly ironic in light of Fodor’s contention that even human thought is a kind of symbol manipulation.’). It wouldn’t do, I thought, to accidentally misrepresent the position of such a towering figure in cognitive science.
And I’m just pointing out that saying so presupposes the answer to the question we’re trying to find.
That gets the implication the wrong way around: if understanding is sufficient for intelligent behavior, that doesn’t mean that it is necessary, and hence, that the mere test for intelligence need not tell us anything about intelligence.
As I pointed out, it’s one of a series of equivalent concepts of randomness and is basically what’s being talked about when a mathematician uses the word ‘randomness’ without any modifier. At any rate, what I said about being able to generate an arbitrary-length, finite amount of randomness via a program (which could just output pre-stored randomness), while being able to prove that no program can produce true randomness, exactly applies to this concept, which is sufficient for my purpose. A ‘Turing test’ for randomness might observe behavior consistent with randomness generation of a given program over any length of time, yet we can say a priori that the program can’t produce randomness. Hence, properties which a behavioral assessment would imply a program possesses can be shown not to be possessed by argument; consequently, behavioral testing is not in general sufficient to delineate the capabilities of programs. An ‘imitation game’ for randomness generation fails, where knowledge of the workings of computers succeeds: in general, we do need to ‘look under the covers’ to see what programs can and can’t do.
I don’t see what that is supposed to do for you at all. Whatever those labels are attached to, to talk about anything definite, the must be attached to something, or you don’t have any reference on the first place and I needn’t even start with the argument. But as soon as they do, I can just permute the objects of that model again, and make exactly the same argument as before. The question is how any utterance is interpreted; to say ‘it’s not interpreted’ solves the problem only at the expense of giving the game away from the outset.
The labels are just there for our convenience, so we can talk about group membership and so on. And again: we don’t talk about labels (except for this kind of meta-discussion). We don’t have access to them at all, if they even exist (whether they do or not probably depends on the substrate). All that is accessible are the relationships, and those don’t change when you relabel things. Swap Object_001 for Object_003, and Chad is still not a cat.
I don’t think I’ve ever said utterances are not interpreted. On the contrary, when an utterance is made, the recipient must do some work to figure out how it fits into their own network of relationships. If two entities share the network above, except for the permutation of the underlying labels, then if one person says “Ajax”, then the other can immediately infer that the object in question is a cat.
In practice, any given utterance is probably ambiguous and there’s likely to be some probability distribution among the possible objects it refers to. This happens all the time with humans and no one has a problem with it, but for a while the two people are building incompatible networks. Eventually (one hopes), enough evidence piles up about the difference and the relationships snap back into place.
But an interpretation in the only way I know how to think about the term is an association of labels to objects in the world, e.g.:
Object_001 → Ajax
Object_002 → Betty
Object_003 → Chad
After all, how else are you going to know what object in the world any of your relations if predicated of?
And then, I just permute those objects again, and your relations end up----with equally much justification as before—picking out different objects.
And if you’re thinking about saying something along the lines of how the structure picks out the interpretation----well, ai hope it’s clear by now that that would just be circular.
All programmers I know periodically reseed the PN counter. Usually from a free running binary counter in the real time clock. Each programmer has his own scheme for what will yield randomness. With reseeding, there is no algorithm the can predict the series of numbers created by the random number generator.
Microcontrollers commonly create a random series by monitoring the current through a Zener diode. The current values are sampled and converted to digital in the A/D converter. The result is not predictable but meets the test for randomness.
These are resources available to the programmer, so they are programming. All testing is behavioral. The only conclusion that can be reached is that the thing tested exhibits the properties defined by the test.
I just want to make a few comments on this bit, and also expand on my own statement. I cannot follow your argument here at all. That last sentence fragment makes no sense. Did you perhaps mean to say “the mere test for intelligence need not tell us anything about understanding”?
At any rate, that last sentence that you quoted me saying, although it expresses two truths, was poorly worded. It’s not so much that understanding “requires” intelligence (although it clearly does) but that understanding is arguably the most important and visible consequence of intelligence, and (not arguably, but definitively) can be tested – and is routinely tested – in the same way.
The Turing test, after all, is a test of all sorts of different levels of understanding: the understanding of language, of the meanings of words, of grammar, and of a vast body of knowledge pertaining to the real world. The “understanding test” that I propose, as I clearly indicated in the previous post, is simply a Turing-like test focused on a subject-specific topic. More generally, this principle of testing for understanding happens in schools and universities millions of times every single day. I’m astonished that this could be in any way controversial.
Well, then I don’t get the relevance of the wiki quote you provided, which has things exactly the other way around, with understanding yielding sufficient capabilities for intelligent behavior. Indeed, that makes much more sense to me—understanding is, after all, just becoming cognizant of the referent of a given sign, i.e. thinking of dogs when met with the string ‘dog’. I’m not sure that this needs intelligence, as such—it isn’t really the outcome of any sort of derivation. Indeed, it isn’t possible to derive, merely from the symbol ‘dog’, that it refers to dogs. I would call it a more primitive faculty of association, perhaps.
We know that most attendants of a typical school are beings relevantly similar to us to find a behavioral assessment of understanding convincing. From there, it does not follow that understanding generally can be accounted for behaviorally. But with something like ChatGPT, there’s no reason for such an assumption.
And in general, it’s easy to conceive of systems passing a behavioral ‘understanding’ check without understanding anything. Consider a system that just produces entirely random responses: every now and then, those will be perfectly adequate. Or consider a system that erases the last half hour of memory of it’s interlocutor, unless they signal that they believe it to understand. Or consider just a great big old lookup table: nobody could hold that a system that does nothing but compare strings to values stored in its database was in any way understanding what those strings refer to—after all, ‘what they refer to’ simply plays no role on the lookup process.
So no, a behavioral check for understanding doesn’t tell us whether a system actually does understand. And of course, again, from ChatGPT’s inner working, we also know that it doesn’t understand. Just as with the lookup table, what a given string means simply is no factor in the completion it arrives at. Finally, if that wasn’t already clear enough, the Newman argument shows that the structural information contained in the language samples simply does not suffice to figure out how the terms of the language match up to any things in the world—the information simply isn’t there.
These are all external inputs. Of course, it’s not a problem to have a computer output randomness you are allowed to input beforehand. But, to quote the great John von Neumann, “anyone who attempts to generate random numbers by deterministic means is, of course, living in a state of sin”.
Your argument seems to be invoking a couple of misdirections. The example of understanding the word “dog” is, indeed, a perfectly valid functional description of “understanding”. It just happens to be a particularly trivial one. But the same functional definition of understanding also applies to the most complex and erudite concepts. The entire gamut can be objectively tested.
On the Wiki quote, I think you’re misinterpreting it. I think it’s axiomatic that understanding anything requires a commensurate level of intelligence (which might be very low, or very high, depending on the subject matter). It’s unfortunate that the Wiki quote uses the term “intelligent behaviour” to describe a consequence of understanding, but what I presume they mean is that understanding something manifests novel behaviours (responses to questions, or whatever) that were not explicit in the original information, but that represent a synthesis of that information into an appropriate set of general concepts. That capability of synthesizing and generalizing information into a mental model is driven by intelligence, not the other way around.
It seems to me that it’s really you who is now begging the question. Implicit in your statement is that an AI cannot be assessed behaviourally because its internal workings are so different from humans. But the whole crux of the argument here is that internal workings don’t matter if the observed behaviours meet the required criteria for whatever we have proposed to measure.
Whether that is the case is the question at hand; just that you keep insisting it’s answered doesn’t mean it is. I’ve given examples of systems passing a behavioral test for understanding that nevertheless don’t understand.
As a bit of a light-hearted digression, this was the result of my seeking ChatGPT’s opinion on the matter under discussion here. I wish the damn thing would stop being so modest and self-effacing. In all seriousness, I’m convinced the thing has been tweaked to avoid alarming the “AI is taking over the world” doomsayers!
Do you possess true understanding, or are you just an automaton mindlessly manipulating symbols?
As an AI language model, I don’t possess true understanding in the way that humans do, but I have been trained to analyze and process large amounts of data and provide responses that are based on patterns and associations within that data.
I do not have consciousness or self-awareness like humans, and I do not have emotions, beliefs, or desires. I am simply a computer program that processes input and generates output based on a set of rules and algorithms.
However, I am constantly learning and improving through the data I am trained on and the interactions I have with users. So while I am not a sentient being with true understanding, I am able to provide useful information and assistance based on the data and knowledge that has been programmed into me.
Would you, therefore, regard yourself as being intelligent in the common meaning of the word?
As an AI language model, I have been designed to process vast amounts of data and perform tasks that would typically require human intelligence, such as language understanding, translation, summarization, and logical reasoning.
In that sense, I can be considered intelligent within the scope of the tasks I am designed to perform. However, it is important to note that my intelligence is based on algorithms and statistical models, rather than subjective experiences, emotions, or consciousness.
I will just say that I’m grateful that we (we humans, like the ones on this board) live in an age where we can be having this kind of debate. At least, I assume most of us are human. (I, of course, am a dog, but I’ve never tried to hide it.)
In all seriousness, I’m convinced the thing has been tweaked to avoid alarming the “AI is taking over the world” doomsayers!