# Strong AI and the Chinese Room
**URL:** https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658
**Category:** Great Debates
**Created:** [December 5, 2002, 6:32am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658 "2002-12-05T06:32:13Z")
**Posts on this page:** 20
**Page:** 6
### Author:  [@TVAA](https://boards.straightdope.com/u/TVAA)
#### Post date: [December 10, 2002, 9:03pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/101 "2002-12-10T21:03:45Z")
**erislover** : Also, Searle seems to forget that just having a brain doesn’t guarantee either a mind or mental contents.
---
### Author:  [@Ramanujan](https://boards.straightdope.com/u/Ramanujan)
#### Post date: [December 10, 2002, 9:14pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/102 "2002-12-10T21:14:39Z")
> [@](#):
>
> I do not see it to be intuitively obvious that the first three axioms are consistent. In fact, A2 seems dubious at best, and question begging at worst.
i actually take issue with each of those axioms. personally, i would very much like to know what searle’s definition of “formal”, “syntactic”, and “semantic”, are. or at least their extensions.
to me, particle a hits particle b, particle a goes in direction a^ and particle b goes in direction b^ seems pretty formal to me. at least in so far as i understand searle’s definition. so, minds are no longer governed my physics. if physics is not formal, how is it that programs must be?
what on earth are “mental contents”?
if A1 and A3 are accurate, certainly computers ought not be able to have internal representations of reality. but i wrote a program once that learned how to play connect four. that sure seemed like a representation of something that wasn’t based solely on the “formal” system of the computer system.
as far as “brains cause minds”, doesn’t it seem as though that’s what he’s trying to show? to claim it as an axiom is just plainly circular.
---
### Author:  [@erislover](https://boards.straightdope.com/u/erislover)
#### Post date: [December 10, 2002, 9:28pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/103 "2002-12-10T21:28:26Z")
> [@](#):
>
> as far as “brains cause minds”, doesn’t it seem as though that’s what he’s trying to show? to claim it as an axiom is just plainly circular.
No… even the Chinese room accepts this. In fact, that’s the point of looking at the Turing test, shall we say, metaphysically: we _already_ attribute this to ourselves. What does it take to attribute it to something else? More than that, when could we say that our ascription was correct? Apparently, if we have a formal description: never. Period. That seems to require a hell of a lot more than three axioms.
None of the axioms examine what meaning is. From what I’ve gathered by glancing at his works at Borders and Barnes and Nobles around the USA, Searle fancies himself as a Wittgensteinian in some way. But if that is the case he has certainly added, or completely changed, what ‘meaning’ means, or what the criteria are for saying something has semantic content.
**Vorlon**
> [@](#):
>
> Also, Searle seems to forget that just having a brain doesn’t guarantee either a mind or mental contents.
Quite!
---
### Author:  [@erislover](https://boards.straightdope.com/u/erislover)
#### Post date: [December 10, 2002, 9:29pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/104 "2002-12-10T21:29:48Z")
grrr… even the _Turing Test_ accepts this. SHEESH.
---
### Author:  [@Azael](https://boards.straightdope.com/u/Azael)
#### Post date: [December 10, 2002, 10:01pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/105 "2002-12-10T22:01:26Z")
So what would pass the Turing test and what might tip you off?
From what I understand of the Chinese Room thought experiment (and admittedly it isn’t much), I see an implication that the “man” is working with algorithims that not not only determine what the question is but formulate an appropriate response - _ **how is that not thinking?** _ It wouldn’t be if the questions had definite answers, but if that were the case then this “Chinese Room” sounds an awful lot like a computer.
Someone show me where I am wrong here?
---
### Author:  [@Ramanujan](https://boards.straightdope.com/u/Ramanujan)
#### Post date: [December 10, 2002, 10:07pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/106 "2002-12-10T22:07:40Z")
> [@](#):
>
> \*Originally posted by erislover \*
> \*\*grrr… even the _Turing Test_ accepts this. SHEESH. \*\*
i don’t necessarily agree with that. my understanding of the turing test is that it provides a method to attribute “intelligence” to something. whether this includes consciousness or a “mind” (whatever that might be), i can’t say.
i can, however, say that i’m rather confident that the validity of the turing test does not rest on the claim “brains cause intelligence” even. i think the claim “i have intelligence”, or “i have [whatever i’m looking for]” is good enough. the method is then based on the fact that we attribute to other humans the properties that we feel we have. not necessarily that all brains have those properties.
---
### Author:  [@Azael](https://boards.straightdope.com/u/Azael)
#### Post date: [December 10, 2002, 10:29pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/107 "2002-12-10T22:29:56Z")
“brains cause intelligence”
hmm…
like only ever having seen birds fly and concluding that wings cause flight only to see a rocket take off?
---
### Author:  [@erislover](https://boards.straightdope.com/u/erislover)
#### Post date: [December 10, 2002, 10:35pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/108 "2002-12-10T22:35:57Z")
> [@](#):
>
> i don’t necessarily agree with that. my understanding of the turing test is that it provides a method to attribute “intelligence” to something. whether this includes consciousness or a “mind” (whatever that might be), i can’t say.
Since it uses beings-with-brains as the standard for declaring when one can validly attribute “intelligence” to a thing, one would imagine that the standard itself is so because it is presumed to have the attribute “intelligence”, eh? 🙂
---
### Author:  [@hazel-rah](https://boards.straightdope.com/u/hazel-rah)
#### Post date: [December 10, 2002, 10:51pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/109 "2002-12-10T22:51:47Z")
> [@](#):
>
> From what I understand of the Chinese Room thought experiment (and admittedly it isn’t much), I see an implication that the “man” is working with algorithims that not only determine what the question is but formulate an appropriate response - how is that not thinking? It wouldn’t be if the questions had definite answers, but if that were the case then this “Chinese Room” sounds an awful lot like a computer.
>
> Someone show me where I am wrong here?
It’s not thinking because it’s just a program that reads some symbols, finds the proper response by consulting x number of lookup tables of x size and then gives back the appropriate response from the tables. There is no intentionality and there is no awareness on the part of the program that the symbols mean anything… as Searle says, they’re not even symbols, because they don’t symbolize anything to the program.
The point of sticking the man in the Chinese room and not letting him do anything except run the programs and consult the lookup tables is to illustrate that the programs and tables don’t have any intentionality… the man may understand English, but he doesn’t understand Chinese!
The linguistic nitpick is that it isn’t even _theoretically_ possible to simulate fluency in a language that way… there is an idea that leaving the logic of the program undefined is the perfect hiding place for something that must be akin to intentionality or semantic understanding, or the proper responses could never be formulated. But that’s a hijack considering we aren’t even characterizing Searle’s position properly yet.
-fh
---
### Author:  [@Azael](https://boards.straightdope.com/u/Azael)
#### Post date: [December 10, 2002, 11:49pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/110 "2002-12-10T23:49:06Z")
> [@](#):
>
> The linguistic nitpick is that it isn’t even theoretically possible to simulate fluency in a language that way
I agree, but what’s more I don’t see how it’s theorectically possible to simulate consiousness or awareness in that way either.
> [@](#):
>
> The point of sticking the man in the Chinese room and not letting him do anything except run the programs and consult the lookup tables is to illustrate that the programs and tables don’t have any intentionality
Doesn’t the man have intentionality (to use the tables and programs to find the appropriate response), and the tables are merely tools to serve that intentionality? And all the components working together form a system that understands Chinese?
I see what you are saying about leaving an element of the program logic undefined in order to leave room for something like semantic understanding but isn’t it more than that? Doesn’t there need to be a response based on more than the symbol itself?
I’m not trying to quibble, just asking questions as they come to me. It’s a pretty interesting subject to try and wrap ones mind around.
---
### Author:  [@Ramanujan](https://boards.straightdope.com/u/Ramanujan)
#### Post date: [December 11, 2002, 12:17am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/111 "2002-12-11T00:17:15Z")
> [@](#):
>
> I see what you are saying about leaving an element of the program logic undefined in order to leave room for something like semantic understanding but isn’t it more than that? Doesn’t there need to be a response based on more than the symbol itself?
“hamburger” is a symbol. “what a hamburger is” is presumably the semantic connection to that symbol. i think if we could define exactly what a hamburger is to you, we would be able to model it symbolically. i’m very curious as to what reasons some give for saying that semantics can’t be modeled symbolically. if the semantic description of something was definable, why couldn’t we describe it symbolically?
---
### Author:  [@foolishmortal](https://boards.straightdope.com/u/foolishmortal)
#### Post date: [December 11, 2002, 12:18am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/112 "2002-12-11T00:18:34Z")
> [@](#):
>
> \*Originally posted by Mangetout \*
> \*\*
> If the Turing test is carried out with a man and woman (as opposed to human and machine) - a scenario that Turing himself mentioned - and we are unable to discern which one is the man, we still can’t assume that the man is a woman, only wistfully admit that we can’t tell the difference. \*\*
Was this a geeky attempt to out himself?
---
### Author:  [@hazel-rah](https://boards.straightdope.com/u/hazel-rah)
#### Post date: [December 11, 2002, 12:41am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/113 "2002-12-11T00:41:08Z")
> [@](#):
>
> i think if we could define exactly what a hamburger is to you, we would be able to model it symbolically. i’m very curious as to what reasons some give for saying that semantics can’t be modeled symbolically. if the semantic description of something was definable, why couldn’t we describe it symbolically?
Well, _Searle_ isn’t saying that. He’s saying that symbol manipulation doesn’t instantly imply semantics is happening. What he isn’t saying is that semantics cannot be modeled symbolically… at some point it has to be for our brains to work! Just that a particular form of Strong AI that manipulates symbols in fact lacks semantics, and therefore cannot be said to understand anything.
-fh
---
### Author:  [@Ramanujan](https://boards.straightdope.com/u/Ramanujan)
#### Post date: [December 11, 2002, 12:49am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/114 "2002-12-11T00:49:18Z")
> [@](#):
>
> \*Originally posted by hazel-rah \*
> **Well, _Searle_ isn’t saying that. He’s saying that symbol manipulation doesn’t instantly imply semantics is happening. What he isn’t saying is that semantics cannot be modeled symbolically… at some point it has to be for our brains to work! Just that a particular form of Strong AI that manipulates symbols in fact lacks semantics, and therefore cannot be said to understand anything.**
if in fact semantics can be modeled symbolically, how does the strong ai that manipulates symbols necessarily lack semantics? what do our brains do that it doesn’t?
my theory is that semantics are all just figments of our imaginations. in a manner of speaking…
and by the way:
> [@](#):
>
> (A3) Syntax by itself is neither constitutive of nor sufficient for semantics.
as posted by erislover, quoted from searle.
my question is, what _is_ sufficient for semantics?
---
### Author:  [@hazel-rah](https://boards.straightdope.com/u/hazel-rah)
#### Post date: [December 11, 2002, 12:55am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/115 "2002-12-11T00:55:17Z")
> [@](#):
>
> Doesn’t the man have intentionality (to use the tables and programs to find the appropriate response), and the tables are merely tools to serve that intentionality? And all the components working together form a system that understands Chinese?
The man has intentionality, but he isn’t using it to deal with the Chinese. He is just running a program and following a set of instructions. This is the contrast… he understands English, but he can not be said to understand Chinese. Your last question is answered by Searle himself in the link in the OP.
> [@](#):
>
> I see what you are saying about leaving an element of the program logic undefined in order to leave room for something like semantic understanding but isn’t it more than that? Doesn’t there need to be a response based on more than the symbol itself?
Yes, but you are making the same point Searle is here… in order to call the symbol processing “understanding” the response has to be based on more than “if given _squiggle squiggle_ reply with _squoggle squoggle_” …those squiggles and squoggles _mean_ something to human Chinese speakers!
My nitpick is that whatever program and lookup tables you use to provide the correct responses to statements in Chinese must contain a degree of program logic and reasoning ability so high that it would in fact be sentience… language ability is separate from thought in the human mind, but you can’t have a conversation without thinking. Daytime talk shows aside.
-fh
---
### Author:  [@hazel-rah](https://boards.straightdope.com/u/hazel-rah)
#### Post date: [December 11, 2002, 1:01am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/116 "2002-12-11T01:01:05Z")
> [@](#):
>
> if in fact semantics can be modeled symbolically, how does the strong ai that manipulates symbols necessarily lack semantics? what do our brains do that it doesn’t?
It doesn’t necessarily lack semantics, it’s just that semantics is not the automatic, inevitable result of an ability to manipulate symbols. According to Searle, that is… not me!
> [@](#):
>
> my theory is that semantics are all just figments of our imaginations. in a manner of speaking…
>
> my question is, what is sufficient for semantics?
Some other module in the brain aside from those modules that deal with language and symbols : )
-fh
---
### Author:  [@foolishmortal](https://boards.straightdope.com/u/foolishmortal)
#### Post date: [December 11, 2002, 1:29am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/117 "2002-12-11T01:29:37Z")
I can’t remember which smart-ass said this, but here it is:
I fell off my bike when I was twelve, and I haven’t been sentient since! \
The point is, if such an injury did occur, neither you nor I would be able to tell.
Likewise, intentionality and semantic content are, as Searle admits, inherently subjective. We grant ourselves sentience largely by instinct and to others by inference. There is no test for intentionality. (or rather, Turing’s the best we got)
\hazel-rah\:
\
… in order to call the symbol processing “understanding” the response has to be based on more than “if given squiggle squiggle reply with squoggle squoggle” …those squiggles and squoggles mean something to human Chinese speakers!
\
Given that the chinese man understands “squiggle squiggle” to mean [semantic content], and the boxed man understands “squiggle squiggle” to mean [must reply with “squoggle squoggle”], what makes the chinese man’s interpretation any more valid? Both are meaningful within the context of each man’s world (China and A Box, respectively).
If I may (massively) paraphrase Focault et al, meaning is arbitrary and specific to the meanee(?).
-ehj
---
### Author:  [@Azael](https://boards.straightdope.com/u/Azael)
#### Post date: [December 11, 2002, 1:37am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/118 "2002-12-11T01:37:32Z")
> [@](#):
>
> Yes, but you are making the same point Searle is here… in order to call the symbol processing “understanding” the response has to be based on more than “if given squiggle squiggle reply with squoggle squoggle” …those squiggles and squoggles mean something to human Chinese speakers!
Hmmm, I agree.
So the program would have to illustrate an understanding independent of the symbols themselves and a failure to do so would result in a failure of the Turing test?
(I realize that the Chinese Room and the Turing test are two different things, I’m just still stuck on a good way of using it)
Could it be that language is after all only the external representation of internal symbols that convey different mental states, ideas and images? If that be so then in order for communication between intelligent beings to happen it requires that both share a common set of internal symbols for any understanding to be taking place.
Where does this leave the humble program? Obviously no formal structure could emulate internal awareness or could it, could an open-ended structure do it? How would we know? Bleh, right back where we started.
Bleh it really is hard to talk about intelligence from a third person perspective.
---
### Author:  [@foolishmortal](https://boards.straightdope.com/u/foolishmortal)
#### Post date: [December 11, 2002, 1:46am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/119 "2002-12-11T01:46:36Z")
> [@](#):
>
> \*Originally posted by Azael \*
> \*\*
> Could it be that language is after all only the external representation of internal symbols that convey different mental states, ideas and images? If that be so then in order for communication between intelligent beings to happen it requires that both share a common set of internal symbols for any understanding to be taking place.
> \*\*
It could, and, IMHO, it is. However I’d replace “common” with “mutually meaningful”, and remove “intelligent”.
---
### Author:  [@erislover](https://boards.straightdope.com/u/erislover)
#### Post date: [December 11, 2002, 2:11am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/120 "2002-12-11T02:11:13Z")
> [@](#):
>
> The man has intentionality, but he isn’t using it to deal with the Chinese.
But that’s like saying, my brain can’t respond in English, so I don’t know English. We can’t actually question the man, so what he knows, or thinks he doesn’t know, is irrelevant. We put a slip of paper in that says, “Do you know Chinese?” and the response is, “Yes” or “Of course”.
> [@](#):
>
> those squiggles and squoggles mean something to human Chinese speakers!
Indeed. But this is the decisive move in the conjuring trick, and it seemed quite innocent. _What_ is meaning to Searle?
I have here the second edition of the Cambridge Dictionary of Philosophy (the purchase of which was inspired by our good man **flowbark** ’s mention; for this I shall be ever greatful). Its exposition of Searle’s conception of language seems to be at odds with the CR, too. It states,
> [@](#):
>
> In Speech Acts: An Essay in the philosophy of Meaning (1969), Searle brings together modified versions of Frege’s distinction between the force (F) and content § of a sentence, and between singular reference and predication, Austin’s analysis of speech acts, and Grice’s analysis of speaker meaning. Searle explores the hypothesis that semantics of a natural language can be regarded as a conventional realization of underlying constitutive rules and that illocutionary acts are acts performed in accordance with these rules. Expression and Meaning (1979) extends this analysis to non-literal and indirect illocutionary acts, and attempts to explain Donnellan’s referential-attributive distinction in these terms and proposes an influential taxonomy of five basic types of illocutionary acts based on the illocutionary point or purpose of the act, and word-to-world versus world-to-word direction to fit.
That is a fucking mouthful. However, handy as this book is, we can head over to Speech Act Theory to get a better glimpse at the above; specifically, we can get a grasp of the intentional, illocutionary acts.
> [@](#):
>
> Beyond intending to produce a certain sequence of sounds forming a sentence in English, a person who utters the sentence ‘The door is open’, e.g., is likely to be intending to perform… (1) the \*locutionary act of saying (expressing the proposition) that a certain door is open, (2) the _illocutionary act_ of making the statement (expressing the belief) that it is open, and (3) the _prelocutionary act_ of getting his listener to believe that it is open. In doing so, he may be performing the indirect speech act of requesting (illocutionary) the listener to close the door and of getting (prelocutionary) the hearer to close the door.
Reading this makes a picture form in my mind of a surfer—intention—, his board—meaning—, and a series of otherwise normal and more or less indistinguishable waves—words and sentences—; specifically, the picture makes me think of the times when the same (or a very similar) sentence can be used in all sorts of different situations and _mean_ (we say) something different each time.
I think this picture of meaning, intention, and sentence (the one I offered or the one I quoted) makes several mistakes. The biggest mistake is that when we try to imagine the meaning of a sentence in isolation, it seems that there is so much going into understanding the sentence that we are stuck making explicit rules before we being. But _so much_ depends on more than just the words, it depends on the situation (stable, or not mentioned, for the CB), it depends on what the listener can in fact do to receive information (the CB has no sight, hearing, touch, taste, or smell: it has a slot which, I think at best, could be said to be an analogous to hearing monotone [not stereo], monotone [single toned] _speech_ [not sound!]), and it depends on what means it has to respond (the box has what would be analogous to a monotone [single toned] voice, and _that’s it_).
Now, imagine a person that has the same means to receive and communicate as the box. What sort of questions could you ask it? I mean, this isn’t even Hellen Keller, for she had touch. Consider, before we get to questioning the box: what could we teach it? —How?
[Previous page](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658.md?page=5)
[Next page](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658.md?page=7)