Is Computer Self-Awareness Possible

Frylock, can you provide some clarity here?

Well, the problem is, in order for cognition to happen on the n-1st level, cognition must first happen on the nth level, because the mental content of the n-1st level entering into perception is dependent on the mental content on the nth level entering into perception, so that the entire infinity of possible levels has to be traversed.

If your seeing something/thinking something/modelling something is dependent on your homunculus seeing something/thinking something/modelling something, then his seeing something (etc.) must in turn depend on his homunculus’ seeing something, and so on. This works quite generally whenever in order to perceive something (say, the meaning of a Chinese sentence), first, a model must be build to be subject to ‘inner’ perception.

Another problem with model-building I personally have is that it can’t have any use. In order to be able to build a model – say, for concreteness, a model of a three-dimensional cube --, all the information that could be gained by having such a model needs to be already present; so actually building the model would be pointless. That is, in order to create an image in the mind of, say, the cube slowly rotating on one of its vertices, the mind needs to already know, or at least be able to derive, how it would look like if the cube spun on one of its vertices – but if that information is already present, creating a model in order to gain access to it is pointless.

That is a simplistic view of an obviously complex situation. I disagree that is a requirement.

Like the problem with the Chinese room - just because an extreme example can be described that has problems doesn’t mean it rules out any other possibility - just that extreme example (and certainly others that are in the same category).

Same thing here: if to realize a model building paradigm, you insert an internal viewer that is external to the model and require it to go through the same process - then yes, you have a problem - but it doesn’t mean that building a model must follow that methodology.

I can think of numerous counter examples. Think of all of the situations in which we think about objects, movement and potential collisions: would you argue that we don’t perform any kind of mental simulation to resolve any of that? I don’t think it’s of the type where each tick of the clock and each bit of movement is accounted for - and it seems like it is heavily handled by pattern matching and parallel processing - but it also seems like there is a model component in which the essence of 3D movement is used to extrapolate future possibilities.

I read his book, and I distinctly remember not pitching it across the room, so I don’t think he really said that. (It was heavy enough so I’d remember doing it.)
My dissertation involved developing a language and compiler for microprogramming, so I’ve got this subject pretty well covered. They syntax of a language involves its structure, like English grammar. I suppose you could consider the architecture of a computer something like its syntax. But syntax is constant - feed a string into a syntax analyzer, and you get an indication whether the string does or does not fit the syntax, and a parse tree. You identify a token in the string as representing a variable or a register, for instance, but you never consider what goes into that variable or register. For the brain, you might say that an experience goes here - but never what that experience is. And, since it appears that our brains keep changing with new experiences, I don’t think you can even do that much. In computer terms, the syntax of an assembly language program says a value goes into a register, and that this value comes from a memory location, but never what that value is, and seldom even the address of the memory location unless it is a constant.

:eek: It is hard enough to bring up a processor when the components are standing still!
What do you call a component? If you are calling it something from a software point of view, like a data item, then they do move. From a hardware point of view, they never move. What does move is electrical waveforms, which are interpreted to represent Boolean values, but these aren’t components, at least not that I’ve ever heard in more than 30 years of working in computer design.

I agree. We are a lot more effective in problem solving than GAs (at least I am.) In the early '70s some AI researchers recorded people solving problems in order to try to figure out how they did it. I don’t remember them having any luck. I suspect we use some sort of massive parallel processing and some sort of analog to content addressable memories used in ICs.
GAs are only useful in demonstrating that computers can be creative, not in reproducing how humans are creative. In any case, building a duplicate of human thought would not be nearly as interesting as building something which thinks in a nonhuman way.

How does model building work in your proposal?

As I said, I don’t think ‘mental simulation’ is really a coherent concept – who is the simulation presented to? Does whoever then create his own simulation to understand the first one? If not, then why was the first one necessary?

Yes, it ‘seems like’. It’s just that computationally, it’s far easier to have it ‘seem like’, than actually having it ‘be that way’. If you or your mind collects some data, and then builds a model from that data, and then based on that model, i.e. some perception of it, executes some action, then, if any step is essentially computable, there is a computable way to derive that action from the data without going through the trouble of creating a model, which in most cases will just amount to computational overhead.

Really, I think you should read Dennett – even if his solutions are not to your liking, I think he is more acutely aware of the problems (of creating a viable theory of consciousness, that is) than just about anyone else I’m familiar with. Some quick link chasing yields only this article online, which I’m not familiar with, but I can’t imagine you’d regret reading his book, Consciousness Explained.

A component of artifact X is a part of X the placement and movement of which is constitutive of the functioning of X.

There are probably water molecules in my desk, but they aren’t components of my desk since their placement and movement isn’t constitutive of my desk’s functioning.

But those electrical waveforms you mentioned fit the bill for a component of the computer.

Recall that this originally came up because I was defending a view that to compile a program is literally (not metaphorically) to build have a machine built inside that computer. Those waveforms are components not only of the computer but of the machines constituted by the programs its running as well. When you get into the grit of the situation–the actual physical stuff that’s happening–you realize what you’ve got is a physical machine made of physical components. The components are often electrons, but they’re no less physical for that.

I actually am not sure I buy the view I’m arguing for here–I’m just defending it against what I thought were some inadequate objections.

It was the thesis of Goedel Escher Bach. (I’m 98 percent sure of that, but it’s been many many years since I read it I’ll admit.)

ETA: Wikipedia, at least, summarizes the book thus:

which looks to me to be a way of saying the book discusses how semantics can be reduced to syntax.

Can you clarify what in the above means that that which understands cannot be defined entirely in terms of syntax?

I’m not certain, but I think Searle thinks at least that a machine that can understand Chinese will need to have hardware designed for language learning built in.

Honestly I’m speculating here though.

Did that address your question?

Well I take it back upon re-reading the wiki quote–its mention of “self-reference” screws up my summary of the summary.

On the other hand I’m pretty sure the article is wrong to use “self-reference” in that way in that sentence. I’m quite sure the book tried to argue that self-reference itself can arise from phenomena definable purely syntactically.

Heh. I actually started re-reading GEB prior to this thread being opened, and in my opinion, how meaning is supposed to arise is somewhat unclear throughout it.

I think that a part of it is that meaning does not lie within any string of symbols – that is only syntax. Rather, meaning lies in mappings between strings of symbols, in the isomorphism that makes the strings ‘I love chess’ and ‘ich liebe Schach’ be the same, or refer to the same state of affairs, in some sense. Without a mapping, an interpretation, any string of symbols is meaningless.

Of course, this is question-begging to some extent: the meaning of one of the strings, at least, must be known in order to interpret the meaning of the other. I think that the hope here is that self-reference may provide an ‘anchor’ for meaning, that in some way, one can use it to ‘bootstrap’ meaning out of meaninglessness, but I’m less clear on how that is exactly supposed to work.

A fair yet difficult question to answer. You will have to give me a little slack on this one as I am only vaguely aware of what this model building might look like.

These are some of my thoughts that shape my thinking about this (and keep in mind this relates more to translating communication, but I thing it is also part of perception):

  1. We have storage of objects/classes of objects/abstract ideas/patterns of motion, etc. that can be retrieved with word(s)
  2. As we try to understand written or spoken communication, while at times there is certainly a parallel aspect in which we perceive and pattern match the whole without much piecing together - there are also times in which we piece together items, retrieving them or activating them or something as the individual ideas are communicated to us until the communication is complete and we have some set of items connected together
  3. The activation or retrieval of thoughts/objects/ideas/etc. is dynamic and flexible - it’s not a simple input/transform/output calculation - they can be held in the current context for a duration and used later
  4. We can take one set of ideas that have been pieced together into a set, and we can take another group, and we can apply them to each other according to the rules they should follow (e.g. these are the players and these are the positions to be filled on the team, put people where their skills match the position - or - this is a ball, this is the process of throwing, put them together to arrive at the desired request)

As our brain pulls together these different elements and temporarily holds onto them for processing and applies the rules of one set to the objects of another set, etc. - this is what seems to me to be “modeling”

Note: for me personally, I feel like I can see these models in my head, when I am programming or designing something, or trying to understand what someone is saying, I am aware of actively manipulating what seems to be 3d “things” in my mind. I couldn’t accurately describe what I see, you would think my description is not very helpful, but that is part of what points me in this direction.

To me that seems like a limiting view of the process - that the simulation is for the benefit of some external party that is not intimately involved in the process itself.

I envision a mind that is not only creating the simulation but is also evaluating the results of the simulation - simultaneously.

It’s the same guy - not some external to that system observer.

I disagree that it is just computational overhead, I think it adds flexibility. But I will agree that the term “simulation” should not be thought of in the same sense that a computer simulation might occur in which every change in state must get calculated.

I think you can have both worlds. The flexibility and general problem solving nature of a simulation like process while at the very same time cutting as many corners as possible for computational speed and efficiency.

If it were a 1 step transformation, would we say “let me think about it”?

When you analyze your own thought processes, aren’t you aware of some times walking through various steps to see the end result?

I will check it out.

I think it addresses it in that you aren’t sure and I need to read further to fully understand his position.

Regarding the hardware designed for language learning built in - that seems surprising because hardware and software are basically the same thing unless he is referring to unique hardware that more closely mimics some feature of the brains hardware.

I don’t think Searle thinks that hardware and software are the same thing. He hasn’t really elaborated on this (that I’ve seen), but I’ve been talking about the “governed by vs following” distinction in part as a way to supply to him a way to make that distinction plausible.

He does like the argument that simulation of something isn’t the same as the thing itself. Hence, he thinks, a perfectly simulated mind isn’t an actual mind. So it looks to me like he doesn’t think that software that’s apparently functionally equivalent to hardware is to be classified ontologically as of a kind with that hardware.

So, if the Chinese room implemented all of this, would there then be understanding? If the virtual Chinese room were implemented within somebody’s mind, would that somebody understand Chinese then?

I think that even most of what you see in the ordinary way is illusion, in a sense – not a virtual picture created as an inner projection, but rather, an absence of facility to note its inadequacies. That’s much more true with inner vision. I made a somewhat lengthy argument to that effect here, if you’re interested.

But then, why does he bother with the simulation? As I said, all that he could gain from it, he must already know to simulate. It’s different in the real world, where actual physical properties of the model exist that are subject to physical laws, so we can, say, build a model bridge and test its stability, but in the mind, those physical laws only exist in so far as they are known, so if we build a bridge in the mind and test its stability, it will hold or fail if and only if we knew that it would beforehand. Even in order to look at it from another side, we already must have known how it would look like from the other side to generate the image.

Indeed I am. But to my thinking, it’s not the self reflecting on a problem that causes the appearance of ‘walking through various steps’ to arise in my mind, it is this (appearance of) ‘walking through various steps’ that causes the self to arise.

But how, if not functionally, are hard- and software to be distinguished? To my thinking, its functionality is the sum total of what characterises a machine of any kind…

This is me and not Searle saying what follows: I think we might be able to draw the distinction by arguing that, contrary to what everyone seems to have assumed in discussions over the chinese room and related affairs, a machine programmed to perfectly simulate a mind isn’t functionally the same as a mind. Crudely put, a machine simulating a mind is governed by a different set of rules than those a mind is governed by. For example, if I type certain keys on the machine, it will behave in a very unmindlike way. There’s no similar rule governing an actual mind. In this sense, the simulation and the reality are of ontologically distinct kinds–different natural laws apply, and there are functional differences between the two, in both the mathematical and the practical sense of “function.” (More to the point–the Chinese Room could and almost certainly will stop acting Chinese-Understanding-Mind-Like at any time, for example when the man inside it gets bored, playful or rebellious.)

Whether this should be a difference that makes a difference for the existence of understanding (much less consciousness) is hard to say. I’d like to argue that it can make that difference but I haven’t gotten around to doing so yet :wink:

That wasn’t supposed to be an exhaustive list of what it takes to create understanding - just some of the types of things I consider when thinking about this problem.

But yes, to the extent the Chinese room performs those types of tricks (and the other ones humans do) in attempting to understand what was communicated and then to determine an appropriate response, I would say yes, it is performing the same tricks that humans do when we say a human understands, which is much closer than a simple input/output mapping. But I can’t give you a definitive answer describing the continuum and explaining where in the continuum understanding arises.

I think I agree, an image in the mind is not really an image, but rather a reference to the components of the image accessed and brought together.

I read that post, some brief thoughts:

  1. I disagree that simulation is computationally not valuable, but I don’t think we are as far apart on this as it may sound - more on this later (don’t have enough time right now)

  2. An interesting visual tidbit that I read about some recent research: did you know they have found that our (well, probably a rats) vision processing anticipates what the visual scene “should be” by activating neurons in the visual processing areas and then it modifies that with the actual information arriving from the sensory nerves - a constant loop of prediction and adjustment

I disagree.

We encapsulate our understanding of some basic rule (much messier and less black and white than this sounds, of course), but we don’t store every combination and permutation of the application of that rule to every object or scenario that could ever happen.

When we apply the rule to an object or scenario that we haven’t experienced, but we do it based on patterns and categories of similar experience, we are performing a calculation that was not known in advance.

For us to know beforehand every situation and scenario, we would have had to store the end result of all possibilities, or we wouldn’t be able to provide an answer - which I know you don’t think we do that.

That may or may not be true in all cases, but I certainly understand and agree with your point to some degree.

But that is a very narrow view of what form modeling can take and for what purposes it can be used.

I’m not sure if I agree or disagree, I need to think about this one.

No one who actually builds computers would agree. A component must be more or less static - signals are not.

You are describing a virtual machine, which back 30 years ago wasn’t as OS based as it is today. A VM has a well defined instruction set, and any program running on the VM has no idea that it is running on the actual physical machine. One of the big sales points of the original IBM 360 series was that it could be microprogrammed to look just like a 1401, which was very popular, so people with 1401 programs could run them without modification on the 360. and then migrate to real 360 code.

I love VMs - they make lots of things easy to understand. I’m not sure how relevant they are in terms of the brain, though, except that if we build a brain simulator it would be a brain VM, and that constructing a simulation of a computer which could reconfigure itself the way the brain does would be just as valid as actually building the computer.

Doesn’t seem that way to me. The part you quoted says that systems can exhibit capabilities beyond those of their components - something I have no problem with. Unless you have some odd definition of syntax and semantics, I don’t see the relevance.

The famous example from Chomsky - Colorless green ideas sleep furiously. No problem with the syntax of that statement, but the semantics are nonsense.

Pretty much any pun is an example.

If you are writing a parser for an arithmetic expression, “a + b” and “a - b” are syntactically very similar - but clearly semantically different. Heck, back when I was writing C code, I wanted to see how you’d write code for an array of pointers to subroutines. I looked at the C syntax, and nearly every way I could think of writing them was legal. Only one worked semantically though.

When Kennedy said “I am a Berliner” at the Wall, the meaning changes dramatically depending on whether you assign the meaning “resident of Berlin” or “doughnut” to the word Berliner.

What is your definition of syntax and semantics, since I’ve never even read of this being an issue