# Strong AI and the Chinese Room
**URL:** https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658
**Category:** Great Debates
**Created:** [December 5, 2002, 6:32am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658 "2002-12-05T06:32:13Z")
**Posts on this page:** 20
**Page:** 4
### Author:  [@DSeid](https://boards.straightdope.com/u/DSeid)
#### Post date: [December 7, 2002, 11:22pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/61 "2002-12-07T23:22:19Z")
Sure a computer can be conscious if only we were able to define what causes such to emerge from the parts.
Some other definitions offered up in the past: (bolding added)
[http://boards.straightdope.com/sdmb/showthread.php?threadid=117034&perpage=50&pagenumber=2](http://boards.straightdope.com/sdmb/showthread.php?threadid=117034&perpage=50&pagenumber=2)
> [@](#):
>
> Science April 22, 2002?
>
> The abstract -
> quote:
>
> Self-Representation in Nervous Systems
> Patricia S. Churchland\*
> The brain’s earliest self-representational capacities arose as evolution found neural network solutions for coordinating and regulating inner-body signals, thereby improving behavioral strategies. Additional flexibility in organizing coherent behavioral options emerges from neural models that represent some of the brain’s inner states as states of its body, while representing other signals as perceptions of the external world. Brains manipulate inner models to predict the distinct consequences in the external world of distinct behavioral options. The self thus turns out to be identifiable not with a nonphysical soul, but rather with a set of representational capacities of the physical brain
>
> Some snippets -
> quote:
>
> Descartes proposed that the self is not identical with one’s body, or indeed, with any physical thing. Instead, he famously concluded that the essential self–the self one means when one thinks, “I exist”–is a nonphysical, conscious thing. At this stage of scientific development, the Cartesian approach is unsatisfactory for three reasons: (i) psychological functions generally, including conscious thoughts such as “I exist,” are activities of the physical brain (1, 2); (ii) aspects of self-regulation (e.g., inhibiting sexual inclinations), and self-cognition (e.g., knowing where I stand in my clan’s dominance hierarchy), may be nonconscious (3); and (iii) as the Scottish philosopher David Hume (1711-1776) realized, there is in any case no introspective experience of the “self” as a distinct thing apart from the body (4). Introspection, Hume concluded, reveals only a continuously changing flux of visual perceptions, sounds, smells, emotions, memories, thoughts, feelings of fatigue, and so forth.
>
> …
>
> This remarkably diverse range of uses of the self-concept motivates [bold]recasting problems about “the self” in terms of self-representational capacities of the brain.[/bold] Doing so deflates the temptation to think of the self as a singular entity and encourages the idea that self-representing involves a plurality of functions, each having a range of shades, levels, and degrees. Further, it broadens the inquiry beyond humans to other species, suggesting that varying levels of coherencing operate in all nervous systems of any significant complexity. The reformulation also sets the stage for designing experiments to determine more precisely the types of self-representations nervous systems have, how they are connected to one another, and the nature of their neural substrates (6).
>
> …
>
> [bold]Self-representations may be widely distributed across brain structures, coordinated only on an “as-needed” basis, and arranged in a loose and loopy hierarchy.[/bold]
>
> …
>
> The most fundamental of the self-representational capacities probably arose as evolution stumbled on solutions for coordinating inner-body signals to generate survival-appropriate inner regulation.
> …
>
> An appealing hypothesis defended by Damasio (12) is that [bold]the self/nonself distinction, though originally designed to support coherencing, is ultimately responsible for consciousness. [/bold]According to this view, a brain whose wiring enables it to distinguish between inner-world representations and outer-world representations and to build a meta-representational model of the relation between outer and inner entities is a brain enjoying some degree of consciousness. … On this hypothesis, the degrees or levels of conscious awareness are upgraded in tandem with the self-representational upgrades.
>
> Interesting point of view, neh?
Thus the self strange looping into itself creates consciousness. Any system that functions with a similar structure can thus be presumed to have some form of consciousness emerging within it.
---
### Author:  [@Mangetout](https://boards.straightdope.com/u/Mangetout)
#### Post date: [December 7, 2002, 11:51pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/62 "2002-12-07T23:51:06Z")
> [@](#):
>
> \*Originally posted by hazel-rah \*
> \*\*What it boils down to for me is that unless you believe in some kind of duality… a soul, a spirit, what have you… then it’s impossible that the human mind cannot be modelled as a machine and that that machine could not be said to “understand” things the same way we do if we modeled it accurately. \*\*
I’m not even sure that the idea of duality is necessarily a huge stumbling block to the idea of AI; if you believe that souls come into being as a result of the growth of a mind; maybe even if you believe that a deity of some sort creates a soul in response to the presence of a suitable receptacle.
---
### Author:  [@Mangetout](https://boards.straightdope.com/u/Mangetout)
#### Post date: [December 7, 2002, 11:55pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/63 "2002-12-07T23:55:12Z")
> [@](#):
>
> \*Originally posted by The Vorlon Ambassador’s Aide \*
> \*\*More to the point, how would you know whether a computer had an inner experience? \*\*
I don’t think there is any way you could, just like there’s no way to tell for sure if I have the same kind of inner life as you or that I see the same blue as you see.
---
### Author:  [@TVAA](https://boards.straightdope.com/u/TVAA)
#### Post date: [December 8, 2002, 12:50am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/64 "2002-12-08T00:50:38Z")
Precisely.
I’d demand that a person who discounts the possibility of AI because computers can’t have experiences to produce proof that they have experiences.
---
### Author:  [@DSeid](https://boards.straightdope.com/u/DSeid)
#### Post date: [December 8, 2002, 1:56pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/65 "2002-12-08T13:56:27Z")
**So what would we require to _presume_ that an AI has a conscious inner experience similar to our own.**
We _presume_ that of other humans because we expereince such an experience ourselves, and other humans act enough like us and are built enough like us, they likely do as well.
The popular way of presenting (but not entirely accurately) Turing’s test would say that if it acts like us (enough that you can’t tell from its behavior that it isn’t) then it must have the same internal experience. A point raised in this thread is that the output of behavior, eg translation of Chinese, does not mean that it is the same internal experience.
So is the nature of that internal experience a function of a seperate soul, or of the nature of the materials we are built of, or of how that material is organized?
If your thought patterns, with all of its the massive nonlinearality and strange loopy self-referentiality, were somehow all exactly replicated in an AI of sufficient size and complexity - would that AI experience your consciousness? Or at least one similar? Would it be you in the machine?
Which is the point of defining what it is about how we are organized that results in a consciousness of our selfs, and then judging an AI sentience not only on outputs but on the organization of the system that produces it.
---
### Author:  [@BioHazard](https://boards.straightdope.com/u/BioHazard)
#### Post date: [December 8, 2002, 3:44pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/66 "2002-12-08T15:44:55Z")
Here’s a question for all of you: How do you know that I am not an AI?
I just started a thread about this:
[http://boards.straightdope.com/sdmb/showthread.php?s=&threadid=148876](http://boards.straightdope.com/sdmb/showthread.php?s=&threadid=148876)
I need to watch my back so partly warmer doesn’t try to kill me in my sleep.
---
### Author:  [@Finagle](https://boards.straightdope.com/u/Finagle)
#### Post date: [December 8, 2002, 4:30pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/67 "2002-12-08T16:30:09Z")
> [@](#):
>
> \*Originally posted by partly\_warmer \*
> \*\*I realized something wasn’t quite fully represented above.
>
> I was speaking about my own actions to indicate that it’s possible for someone to do something to stop the situation you describe. In practical fact, I doubt I’d be involved – for various reasons.
>
> In practice, I would expect environmental, safety, and religious groups to stop artificial life that couldn’t be distinguished from real life. I simplified the argument by pointing out that one person alone, even, is capable of acting.
>
> What would happen if they all failed (as perhaps you are hoping and assuming will happen)? The problem – for some mad scientist – is that artificial life wouldn’t happen all at one time. Limited artificial life will come out first. As the problems with it are discovered, research will be stopped. It will never get to the point where a human is completely indistinguishable from an AI mechanism.
>
> \*\*
```
Golly. I'd expect someone who'd spent time as an AI project manager to be able to understand the difference between Artificial Life and an AI program. Given the amount of hardware needed to support an artificially intelligent entity, it's not at all likely that one could spread in an unrestricted fashion; bad sci-fi movies to the contrary. There are technologies (nanotechnology and genetic manipulation) that pose this kind of problem, but they are unrelated to the current discussion.
```
The biggest problem with artificial intelligence is that it could displace knowledge workers, but again, given the level of hardware required – not too likely in the near future.
To get back to the OP, the primary flaw in Searle’s Chinese Room argument is that it assumes it’s actually possible to create the “lookup tables” that convert from English to Chinese in a mechanical fashion. There are an infinite number of possible inputs, few of which correlate one to one with a Chinese utterance. So unless the intermediary can reason about the input and generate a reasonable response, you can’t, even in theory, build the system that Searle proposes.
Take, for example, the famous translation of the Coke slogan “Coke adds life”. I believe it translated, more or less, to “Coke will raise the spirits of your dead ancestors”. In order to get a _reasonable_ response, you’d have to reason about what the advertising slogan meant to say, and find a corresponding utterance in the Chinese culture. This is a _creative_ endeavor, as anyone who has ever tried to translate imagery from one language to another would attest. Not only is it creative, but there’s no one unique answer. This pretty much eliminates the lookup table approach.
---
### Author:  [@TVAA](https://boards.straightdope.com/u/TVAA)
#### Post date: [December 8, 2002, 6:49pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/68 "2002-12-08T18:49:14Z")
People are programmed to consider certain signals to be evidence of “humanity”.
Our visual system is designed to identify faces as categorically distinct. More interestingly, robots designed to simulate human facial expressions typically cause people to respond to them as if they could understand what was said to them (verbally and nonverbally), despite the robot having absolutely no ability to process information in non-expressive ways.
I suspect that humans are fundamentally unable to accept computers/robots/programs as either “alive” or “intelligent” because they don’t have the characteristics that we instinctively associate with either life or sentience.
---
### Author:  [@BioHazard](https://boards.straightdope.com/u/BioHazard)
#### Post date: [December 8, 2002, 9:45pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/69 "2002-12-08T21:45:11Z")
> [@](#):
>
> \*Originally posted by Finagle \*
> \*\* To get back to the OP, the primary flaw in Searle’s Chinese Room argument is that it assumes it’s actually possible to create the “lookup tables” that convert from English to Chinese in a mechanical fashion. There are an infinite number of possible inputs, few of which correlate one to one with a Chinese utterance. So unless the intermediary can reason about the input and generate a reasonable response, you can’t, even in theory, build the system that Searle proposes.
> \*\*
There is not and infinite number of possible inputs or outputs if the input uses proper grammer and word usage, and if all nonsense inputs are avoided. To lesson the vast number you could use a keyword look up system based on what keywords and the combinations of them. Granted, the number is very, very large and it would take the person a very long time to look it up. If he cant find it, he could pass something out which is the translation of “I don’t understand”. You could even make a lookup table for those some errors based on key words.
---
### Author:  [@robertliguori](https://boards.straightdope.com/u/robertliguori)
#### Post date: [December 8, 2002, 11:01pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/70 "2002-12-08T23:01:52Z")
> [@](#):
>
> Take, for example, the famous translation of the Coke slogan “Coke adds life”. I believe it translated, more or less, to “Coke will raise the spirits of your dead ancestors”. In order to get a _reasonable_ response, you’d have to reason about what the advertising slogan meant to say, and find a corresponding utterance in the Chinese culture. This is a creative endeavor, as anyone who has ever tried to translate imagery from one language to another would attest. Not only is it creative, but there’s no one unique answer. This pretty much eliminates the lookup table approach.
And? You could create a symbolic lookup table that translates words into symbolic images, and another to turn those images into another language. Heck, you could cross-link images with each other, to express “deoxyribonucleaic acid” as “double helical chain of neucleotides held together by deoxyribose”, if there was no direct Chinese translation for DNA.
---
### Author:  [@Ramanujan](https://boards.straightdope.com/u/Ramanujan)
#### Post date: [December 9, 2002, 12:02am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/71 "2002-12-09T00:02:59Z")
> [@](#):
>
> \*Originally posted by Finagle \*
> **To get back to the OP, the primary flaw in Searle’s Chinese Room argument is that it assumes it’s actually possible to create the “lookup tables” that convert from English to Chinese in a mechanical fashion. There are an infinite number of possible inputs, few of which correlate one to one with a Chinese utterance. So unless the intermediary can reason about the input and generate a reasonable response, you can’t, even in theory, build the system that Searle proposes.**
let’s be careful not to misrepresent the argument presented. it is, after all, a _gedankenexperiment_. such a system of lookup tables built out of books or a digital program might be impossible in terms of practicality, but those are just the sorts of issues we seek to ignore in formulating thought experiments.
there are a finite number of english words and symbols, and a finite number of chinese words and symbols, so the number of combinations possible is finite, for all sentences of finite length. yes it would take a while, but again, that is an issue that is not relevant.
searle would maintain his argument if we used more optimized versions than lookup tables, such as an ANN to interpret symbols determined by a hash table lookup. hidden markov models are generally used to solve problems such as these, too, and can provide “mistakes”, since a human would not believe another human was infallible.
so, no, it’s not impossible. not easy, to be sure. but it’s possible, and the practicality of the situation is not relevant.
---
### Author:  [@Finagle](https://boards.straightdope.com/u/Finagle)
#### Post date: [December 9, 2002, 3:54am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/72 "2002-12-09T03:54:42Z")
> [@](#):
>
> \*Originally posted by Ramanujan \*
> \*\*let’s be careful not to misrepresent the argument presented. it is, after all, a _gedankenexperiment_. such a system of lookup tables built out of books or a digital program might be impossible in terms of practicality, but those are just the sorts of issues we seek to ignore in formulating thought experiments.
> \*\*
Well, it is true that you could have a finite set of utterances, given a finite set of symbols and sentences of a finite length. But you can’t depend on a finite sequence of utterances and replies. The same utterance will require a different reply depending on context. So your lookup table would have to be context sensitive (I’m probably not using this in the strict formal language sense) in that for each utterance, you’d have to look up the entire previous discourse. So either you limit the length of the discourse (which makes the argument completely artificial) or you’re proposing something that isn’t even theoretically possible. Either way, I think you have a straw man.
Note that if you add sensory input into the mix (which most AI researchers would agrees is necessary), then you start dealing with fuzzy, noisy analog data (e.g. floating point numbers) so the idea of a finite grammar isn’t even remotely possible.
Searle’s \* gedankenexperiment \* argument proposes an algorithm that is not only impossible to actually implement but one which no sophisticated researcher would ever propose for implementing an AI system. So arguing that the mechanism that implements this algorithm is not intelligent is pretty vacuous.
---
### Author:  [@hazel-rah](https://boards.straightdope.com/u/hazel-rah)
#### Post date: [December 9, 2002, 3:59am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/73 "2002-12-09T03:59:49Z")
I think that was what Searle was trying to illustrate when he wrote about the Chinese room… that language cannot be reduced to a manipulation of symbols.
---
### Author:  [@robertliguori](https://boards.straightdope.com/u/robertliguori)
#### Post date: [December 9, 2002, 4:16am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/74 "2002-12-09T04:16:06Z")
Huh? How was that shown?
---
### Author:  [@matt\_mcl](https://boards.straightdope.com/u/matt_mcl)
#### Post date: [December 9, 2002, 7:36am UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/75 "2002-12-09T07:36:24Z")
> [@](#):
>
> There is not and infinite number of possible inputs or outputs if the input uses proper grammer and word usage, and if all nonsense inputs are avoided.
Certainly there is.
Bob laughed.
Bob laughed and Lily cried.
Bob laughed, Lily cried, and Ted jumped.
Bob laughed, Lily cried, Ted jumped, and Erica gave blood.
Bob laughed, Lily cried, Ted jumped, Erica gave blood, and Edward laughed too.
Bob laughed, Lily cried, Ted jumped, Erica gave blood, Edward laughed too, and Fraser watched TV.
and so forth.
That was one of the ways that Chomsky demonstrated that behaviorism was insufficient to account for human linguistic behaviour.
---
### Author:  [@erislover](https://boards.straightdope.com/u/erislover)
#### Post date: [December 9, 2002, 2:34pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/76 "2002-12-09T14:34:56Z")
> [@](#):
>
> I think that was what Searle was trying to illustrate when he wrote about the Chinese room… that language cannot be reduced to a manipulation of symbols.
I would like Searle to teach English to someone with no eyes, no hands, no ears, no smell, and no sense of touch. Until such time, I remain convinced that the Chinese Room understands Chinese as much as anyone does: when questioned, it gives the proper response. That is, more or less, all I have to go on for everyone else.
Attributing consciousness when consciousness is defined as “inner experiences” is quite irresponsible. It would be the equivalent of seeing one black swan and asserting with great confidence—near perfect certainty—that all swans are black. And I suppose, in a degenerate sort of way, if one could never see a different swan than this it would be true in a solipsistic way…
---
### Author:  [@erislover](https://boards.straightdope.com/u/erislover)
#### Post date: [December 9, 2002, 2:36pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/77 "2002-12-09T14:36:10Z")
> [@](#):
>
> That was one of the ways that Chomsky demonstrated that behaviorism was insufficient to account for human linguistic behaviour.
😕 Can you expand on this demonstration? I don’t see it as immediately obvious.
---
### Author:  [@BioHazard](https://boards.straightdope.com/u/BioHazard)
#### Post date: [December 9, 2002, 5:48pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/78 "2002-12-09T17:48:33Z")
> [@](#):
>
> \*Originally posted by matt\_mcl \*
> \*\*Certainly there is.
>
> Bob laughed.
>
> Bob laughed and Lily cried.
>
> Bob laughed, Lily cried, and Ted jumped.
>
> Bob laughed, Lily cried, Ted jumped, and Erica gave blood.
>
> Bob laughed, Lily cried, Ted jumped, Erica gave blood, and Edward laughed too.
>
> Bob laughed, Lily cried, Ted jumped, Erica gave blood, Edward laughed too, and Fraser watched TV.
>
> and so forth.
>
> That was one of the ways that Chomsky demonstrated that behaviorism was insufficient to account for human linguistic behaviour. \*\*
Look at it in the keyword scheme:
Bob laughedTed jumped Erica gave blood [Edward laughed] {too}(, and) Fraser watched TV
So even in that pyramid you gave as an example it all boils down to:
[Bob laughed]
(,)
[Ted jumped]
[Erica gave blood]
[Edward laughed]
{too}
(, and)
[Fraser watched TV]
(.)
Extend this and you will eventually have the beginings of a usable, finite system.
---
### Author:  [@Ramanujan](https://boards.straightdope.com/u/Ramanujan)
#### Post date: [December 9, 2002, 6:08pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/79 "2002-12-09T18:08:02Z")
> [@](#):
>
> \*Originally posted by matt\_mcl \*
> \*\*Certainly there is.
>
> Bob laughed.
>
> Bob laughed and Lily cried.
>
> Bob laughed, Lily cried, and Ted jumped.
>
> Bob laughed, Lily cried, Ted jumped, and Erica gave blood.
>
> Bob laughed, Lily cried, Ted jumped, Erica gave blood, and Edward laughed too.
>
> Bob laughed, Lily cried, Ted jumped, Erica gave blood, Edward laughed too, and Fraser watched TV.
>
> and so forth.
>
> That was one of the ways that Chomsky demonstrated that behaviorism was insufficient to account for human linguistic behaviour. \*\*
actually, if you stipulate that sentences be finite in length, you are quite incorrect in your account.
the number of sentences may be obscenely large, but it boils down to all finite combinations among a finite number of finite sets, of which there are finitely many.
so you would eventually run out of things to add onto your sentence.
---
### Author:  [@erislover](https://boards.straightdope.com/u/erislover)
#### Post date: [December 9, 2002, 6:27pm UTC](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658/80 "2002-12-09T18:27:59Z")
To be technically correct, it would be finite permutations. The order is important, or conceivably could be important, given the methodological rule with which one was constructing the sentences. Of course, given such a methodlogical rule, we find ourselves smack dab in a behavioristic description, even if we are not asserting behaviorism proper, so I’m going to try and find Chomsky’s refutation of behaviorism.
[Previous page](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658.md?page=3)
[Next page](https://boards.straightdope.com/t/strong-ai-and-the-chinese-room/140658.md?page=5)