(Full text accessible here. If you can’t access that page, a preprint version is also available at my Academia.edu-page.)
I’m not going to try and give a detailed description of the arguments in the paper. Instead, I want to present a brief summary of its main claims; if there’s any interest in any of the points, we can then dive into those more deeply. So, here goes:
The stuff of the world is physical stuff bearing relational structure. Structure here means essentially how things—elements of a mechanism, components, parts of an aggregate—relate to and interact with one another. Structure, at some level of granularity, is shared between an object and a model of that object—for example, the solar system and an orrery. Or more aptly, a map and the territory it represents.
Computers are objects acting as models of the structure of abstract computations (as formalized, e. g., by Turing machines). A concrete computation—say, the function of adding two numbers—is instantiated by means of interpreting this structure. To be a model of something, a system needs to be interpreted—to be intelligible, every map needs a legend. This interpretation is furnished by means of the intrinsic properties of ‘physical stuff’ (the ‘structure-transcending’). Structure itself radically underdetermines its domain; hence, minds need a non-structural—and therefore, non-computational—element to perform interpretational acts.
These structure-transcending mental properties are nothing but the intrinsic nature of physical stuff, which physics itself is silent on (the ‘inner un-get-atable nature’ of matter, in the words of Arthur Eddington). These can be brought to light by means of a certain self-referential process, paralleling von Neuman’s design for a self-reproducing automaton. Such a design is self-reading, and acquires a dependency on its intrinsic properties as a result. Due to its evolvability, it comes to resemble—to model—an organism’s environment.
This is the origin of subjective experience: a structure impressed on the mind by the senses, capable of reading itself, and thus, leapfrogging structural underdetermination due to access to its own intrinsic properties (filling the ‘form’ of structure with the ‘content’ of its own intrinsic nature). It’s also the origin of the difficulty of explaining subjective experience (the ‘Hard Problem of consciousness’): the intrinsic can’t be derived from the structural, and hence, no models of it can exist.
The idea in that paper has close ties to some of my other works—the von Neumann process alluded to above is the core of my account of intentionality (the ‘other problem’ of the mind), first presented in this article (academia-link), and elaborated here (academia-link).
Also, the underlying ontology suggested fits very well with my reconstruction of quantum mechanics, published here (academia-link).
Finally, the notion of modeling is an important element to my suggestion of how artificial intelligence can illuminate some key notions of Buddhist philosophy (see here).
All of this is just for context—although should anybody find any of those subjects interesting, I’d be happy to discuss them.
A very interesting paper. I enjoyed reading it this morning over breakfast. It reminds me of an implementation issue when working with artificial intelligence. It isn’t sufficient to say “The AI giving correct answers and therefore it is working correctly.” It is all just 1s and 0s after all, so there is a risk that we are interpreting the outputs in a particular way that gives the impression that the algorithm is working correctly. To confirm that the AI is, in fact, computing properly, what we will often do is give it some known cases, and ensure that it is processing them properly; i.e., try to understand how it is reaching its conclusions. Sometimes this cannot be done. The danger is that even if it seems like an AI is working properly if it is our interpretation of the output that is doing the work, then there is the possibility that some future samples will fail.
I’ll be upfront here and say that most of this is way above my pay grade, so I’m probably getting this wrong.
Anyway, to use your map example, people have programmed computers to create quite detailed maps in video games, arbitrarily close to a real “world” inside the computer. I don’t think I understand why a computer couldn’t be used to map out a brain to arbitrary precision, creating consciousness (whatever that is) inside the computer. Since everything a computer does is ultimately computation, this consciousness is computation.
One could say that climate and weather aren’t computation, they’re structure, but computers are getting more and more accurate at modelling both. And, they may be more complicated to model than a brain – a neuron seems to be more modelable than a weather system, so modelling a brain is “just” a matter of scaling that up, rather than dealing with chaos, sensitive dependencies, and so on.
I haven’t read your article (work is insane right now, so I’m just getting in some SDMB time before things kick off), so I’m just working off of your brief summary here.
Anyway, I look forward to being torn apart. I’ll try and return to the thread later to accept my lumps!
Yes, that’s a related issue—connected, I believe, to the notion of explainable AI (do correct me if that’s the wrong tree to bark up). Essentially, you want what I call a model, in order to be sure that the AI’s behavior, its output, is actually related to what we expect its output to be—even if it prints out ‘cat’ when given a cat-picture in the vast majority of cases, that doesn’t necessarily mean that it reacts to a four-legged furry animal with whiskers, but perhaps, all of the cat pictures were taken under certain lighting conditions, for example.
Well, let’s unpack this into two parts. First, I assert that structure underdetermines content; that’s known as 'Newman’s problem’, as raised by the mathematician Max Newman. It’s a bit of a technical issue, but basically, if you have some structure (say, a set of relations) fulfilled by a set of N objects, then each set of N objects fulfills that structure, and hence, merely specifying a structure really only tells you the (minimum) cardinality of a domain. Mathematically, it’s a trivial consequence of the powerset axiom—each set of relations is just a particular grouping of subsets of the domain, and the powerset axiom assures you that with the original set of objects, every such grouping exists, you just have to pick one.
It turns out that also non-set theoretic notions of structure are plagued by this problem, but I won’t go into that.
The second part is that a given computer instantiates a computation by essentially instantiating the right structure. This I motivate by an intuition pump, where I show that the same device can be interpreted as implementing different functions (the ‘parable’ in section 2.2). Essentially, nothing about a physical system has any intrinsic connection to, say, numbers; hence, a computer that computes some function of the natural numbers (and all computable functions can be understood as functions of the natural numbers, for concreteness even functions taking n-tuples of numbers to a single number output) can only instantiate the structure of any given function. This is what I use in my example to argue that for different users, one and the same object can instantiate different computations.
Now, the question arises how a given computation is instantiated for any user. This is the question of implementation, and I appeal to a specific notion—the abstraction/representation account, due to Horsman et al—to make sense of this question. This account depends on a representation relation that connects physical systems with abstract objects; that’s my main target. Basically, the conclusion then follows by noting that since structure can never yield a definite computation, and hence, give a definite answer to the question of what any given system computes, we need to appeal to non-structural properties in order to implement a computation. But if a system instantiates a computation by structural equivalence, then these non-structural properties can’t be computational.
The problem is more basic than that. What a computer, in the most widespread realization, actually does has nothing to do with maps, or worlds, or brains; it’s just voltage levels being shunted around. The question then becomes how these voltage levels relate to all of the things that we take a computer to compute—that’s the question of implementation.
We’ve carefully designed modern computers to present us with contact surfaces that make it ‘obvious’ what’s being computed. But think back to the past when computers were massive cabinets covered in blinking lights, outputting cards with a certain pattern of holes cut into them: what, exactly, does such a system compute? If you find a box where certain lights light up in response of certain buttons being pressed among the remnants of an alien civilization, how would you figure out what it computes?
I use an example in section 2.2 to argue that this question doesn’t have a unique answer: one and the same system can be interpreted as implementing different computations. But if that’s true, then the faculty of interpretation itself can’t be computational—that would entail an infinite regress: who interprets the brain as implementing the computation that interprets the box in front of me as computing the sum of two numbers? And so on.
It’s a very hard thing to do, to become aware of the symbols that our computers output as symbols, rather than as the things they mean. But once one does, it becomes apparent that it’s not actually the case that one can simply say ‘this computer adds two numbers’; whether it does anything connected to numbers at all is an interpretational question. How this interpretation works, then, is essentially the topic of my paper.
Maps aren’t perfectly isomorphic to the territories they represent.
That’s kind of their whole deal.
A computation that is perfectly isomorphic to the firing of neurons in a human brain isn’t a “map” in that sense. It’s structurally the same thing as the original. Interpretation is superfluous here. It’s an added element, where nothing need be added.
You’re adding extra stuff here, and there’s no compelling reason that I personally see to believe it’s necessary.
I really don’t know why you think “interpretation” is necessary. Though I didn’t participate, I read a great deal of the previous thread where you elaborated on this, and this has always been my sticking point.
I’m glad you posted this stuff. You had mentioned the paper in a previous thread and had me curious about it ever since. I’m going to read it now, so my question below might be answered by reading it.
(bolded and font sized)
I remember your point from previous thread (lights on the box could represent multiple computations), and I’m following what you posted up until the bold/italic sentence.
Are you referring to the interpreter’s physical stuff (e.g. a humans brain cells)?
Are the intrinsic properties known or guessed at? Or is it just clear that there must be some critical intrinsic properties because structure alone has been ruled out?
Thanks! This is what I was clumsily trying to get at. I’ll bow out and let you and him argue. I didn’t understand the interpretation part at all – some sort of anthropic thing? (OP, no need to answer my questions, because I’m still pretty at sea here, but I’ll follow along as best I can)
Summary for those reading that didn’t read the previous thread (hopefully this is close enough): HMHW described a box with input (switches) a circuit of some sort and output (lights). He listed a pattern of inputs and corresponding outputs that mapped to a specific function, implying the box is “computing” that function. He then described a second function that corresponded to the same set of inputs and outputs and that implied it is also “computing” the second function.
The argument is then that an external observer must interpret which function is actually being computed, the switches+circuit+lights alone are not enough for us to identify which function is being computed. When applied to consciousness, it means there must be something interpreting the computation to determine that it’s the “consciousness” computation that just happened as opposed to some other computation, but the computation itself can’t be that thing that does the interpreting.
Question for you Hellestal:
Why do you think an interpreter is not necessary?
For the switches and lights problem, given your description, there’s no way for the interpreter to figure out which function its computing. It seems like a strange example anyway – if there’s no way even in principle to figure out which function, then both functions are the same in reality. If there is a way to figure out which function is being calculated, but the limited number of inputs and outputs is preventing that, then that’s just an interface problem, not some fundamental issue that needs an interpreter.
For example, if you had a Turing test setup where the questioner could only ask:
What is your name?
How old are you?
It would be impossible to tell whether the entity responding was really intelligent or not. But, a wide-open Turing test would be much tougher to pass. In the example you gave, if more switches and lights would reveal which function, than that’s a problem of setup, not a problem of intelligence.
For the interpreter question, not directed at me, why is it needed? Does a hermit who completely cuts herself off from the rest of humanity no longer count as intelligent, since there’s no interpreter? (Maybe since the hermit is conscious, she can interpret herself, but maybe so can an intelligent computer. To say otherwise seems like question begging to me).
Sorry I keep poking my head in and then promising to leave. This thread really interests me. Also, congratulations to the OP for getting the article published!
Take the following example: consider
[ol]
[li]The set of books on your shelf, ordered by thickness[/li][li]The set of your maternal ancestors, ordered by ancestry[/li][/ol]
They’re both structurally exactly isomorphic: the structure they possess is a linear ordering relation. There’s additional stuff there, but we can abstract away from that—I don’t presume, for instance, that you hold that a brain simulation must simulate every single atom, or down to the quantum field/string level.
Yet, that structure, the ordering relation, despite being exactly the same across both sets, can be instantiated in very different ways. The rocks in your yard, by size. Your neighbors, by age. And so on: the same structure, different objects.
You can use the books on your shelf as a model of the set of your maternal ancestors. If you know that The Old Man and the Sea is mapped to Ethel, and Moby Dick is mapped to Delilah, you immediately know that Delilah is an ancestor of Ethel, since you know Moby Dick is heftier than The Old Man and the Sea.
But there’s nothing that intrinsically makes the set of books on your shelf a model of your maternal ancestors—nothing safe your interpretation. You can equally well interpret it as a model of your neighbors’ ages.
The same relation obtains between a computer and the computation it implements. I think the example I give in the paper (in section 2.2) makes it clear; if you want, I can repeat it, and we can discuss it in detail. But the gist is that the structure of switches flipped and lights lit has exactly as much claim to implementing the function of addition, as it has to implementing any of the other functions associated with the device.
There is a relation of switches and lamps containing, for example, the element (up, down, down up, off, on, on); the same structure is realized by the function of addition, where that element is mapped to (2,1,3), but just as well by any of the other functions, where the element is mapped to (1,2,4) or (1,2,6).
Or we can come at this from the other end. Take the implementation relation as given by the A/R-account—see section 2.4 in the paper. Each physical state of a system is mapped to an abstract object; the physical evolution of the system then induced a corresponding evolution in the abstract space—i. e., the computation. This mapping depends on the theory used to describe the system; different theories yield different mappings. But what theory we use to describe a system isn’t uniquely fixed by the system (the famous ‘underdetermination of theory by data’). Hence, different theories will yield different computations. I give a concrete example of different theories yielding different computations in that section.
Well, how would you go about finding out what an alien artifact computes? Say you find the box I describe somewhere. You can use it, evidently, to compute the sum of two inputs. You can also use it to compute one of the other functions I specify. How do you decide who’s got it right?
Well, I’m referring to the intrinsic properties of the interpreter’s physical stuff. The intrinsic properties are what support the relational structure. See, whenever we probe some element of the world, all we can know is how it reacts to certain stimuli—roughly, its input-output behavior. But different things can bear the same input-output behavior—that’s what the great Arthur Eddington meant when he claimed that science is silent on the ‘inner un-get-atable nature’ of matter. It’s also at the heart of Bertrand Russell’s philosophy of science, nowadays known as structural realism.
Hence, we know the relations of stuff, but not the relata—the stuff that stands in those relations. But if all we knew were relations, than all we’d really know would be cardinalities—all mere relation tells us are assertions of the form ‘there are N things’ (this is again Newman’s problem). To ground these relations, something that goes beyond structure—the ‘structure-transcending’ properties of Strawson—is needed. Those are the intrinsic properties of (in the end) our brain-stuff.
These intrinsic properties, in my model, are known by virtue of the von Neumann process. Von Neumann constructed a blueprint for a self-reproducing machine that incorporates its design within itself—it consists of a machine, together with its blueprint on a tape, such that it can first read the tape and construct a copy, then merely copy the tape.
This avoids certain self-referential paradoxes—one can, in fact, show that a machine can never, for instance, scan itself to create a copy of itself. Now, this device can evolve; and the idea is that in its evolution, it essentially acquires information about its environment—the same way that a dolphin’s streamlined form contains information about its aquatic habitat. However, the environment, in this case, is essentially the data acquired from the sense organs, and in that way, the von Neumann replicator comes to be a model for that data, and with it, the environment itself.
Now then comes the trick. You can think of the structure and the intrinsic properties as somewhat akin to a set of axioms, and a concrete realization thereof (a model)—for example, the Peano axioms and the natural numbers. You’re probably familiar with the phenomenon of Gödel incompleteness: essentially, the axioms don’t fully specify the model, and hence, many models of a given axiom set are possible—this being the analogy of the Newman problem.
The Gödel sentence, itself, is then true in some models, and false in others—that is, it depends on the model itself, on the intrinsic properties, in the analogy. It does so by, in a sense, containing a copy of itself—in the same way as the von Neumann construction does (in fact, the von Neumann construction and the Gödel sentence are both instances of a deep theorem first formulated by William Lawvere). Hence, the von Neumann construction—or so I argue—, in the analogy, can acquire a dependence on the intrinsic properties, and by means of this dependence, instantiate an interpretation (this is basically in section 3.6).
Then, by the exact same reasoning, there’s no way of figuring out what your calculator is computing. That it computes, say, ‘addition’ depends on a specific way of interpreting lights as numbers—and with a different interpretation, any of a number of other functions can be equally as well associated to the calculator.
Well, but they’re not—the set of computable functions can be described as functions from n-tuples of natural numbers to natural numbers, and as elements of that set, they’re distinct.
The example is completely generalizable, by the way—adding more switches and more lights will not lead to narrowing down the possible functions (in fact, the possibilities will increase combinatorially).
It’s more like: you can ask each and any question you like; however, you can never be sure if the machine is in fact speaking English. The problem is exactly equivalent to the one of finding a dictionary with definitions entirely in an unknown language: even though every word is in there, and explained entirely, you’ll never be able to decipher it.
I should note one thing: I don’t dispute that an intelligent, even conscious, machine is possible. However, its consciousness—entirely like our own—won’t be due to the computations it performs; rather, the computations it performs—as in, the definite functions it instantiates—will be due to its conscious experience, its capabilities of interpretation, and indeed, self-interpretation.
No need to be sorry at all. Discussing this is, after all, what I posted this for.
Can we focus for a minute on the input/output behavior?
If two machines output the same results for the same inputs, then, for that interface, they are identical regardless of what’s going on behind the scenes. Give one input switch and one output light, you can’t tell whether it’s a simple circuit or a team of geniuses staring at the state of the switch and turning the light on when the switch moves.
However, if you have a wide-open set of inputs and a wide-open set of outputs, and they still put out the same results, then, for all intents and purposes, they are functionally equivalent. What difference does it make whether the underlying substrate is neurons or silicon (or, whatever follows silicon to make the great leaps needed for hard AI)?
ETA: Just saw your most recent post. Reading it now.
if you only look at input+output, and you have a pre-computed table of all possible inputs (which includes internal state) that produces the exact same output as some human (that we assume is conscious), then would you say that this lookup table machine is conscious also?
This is that Chinese Room thought experiment all over again.
I don’t think it’s possible, even in principle, to have all the inputs mapped to all the outputs, because it’s a combinatorial problem that would quickly require a computer bigger than the universe itself. It would have to be essentially infinitely recursive to deal with sentences like, “my brother’s gardener’s sister’s aunt…” There are effectively an infinite number of possible sentences. And, that’s leaving aside the infinite number of visual cues that we have to process as well. And, the pre-computed table would have to have information about itself, and information about that as well, etc.
Algorithms, though, aren’t pre-computed tables, so I’m not sure how this is relevant.
Okay, thanks for the explanations above, Half Man Half Wit.
I still have some questions, but let me explain my manner of thinking.
Imagine a complicated device. It could be anything, but to give a concrete example, this device finds loose change on the sidewalk and puts $20-bills on my kitchen table. It’s a computer in the general sense of taking inputs (the environment around it), doing a computation (manipulating its state and the environment), and returning an output (physical objects placed into the environment).
The device has a bunch of components to do different things: sense its environment, move about, identify objects, pick things up, count money, exchange coins for bills at a change machine, etc. They are also computers, since they take inputs, do computations, and return outputs. The components communicate with each other and must interpret the outputs of each other.
Now let’s examination this device. Do we need a non-structural interpretation to understand the output of this device? I say no. The $20 bills that come out have no meaning beyond their own existence. I say the output is inherently structural.
But let’s look inside the machine. The components are interpreting each others’ outputs. Is that non-structural? Maybe. That is getting into Searle’s Chinese room, but it’s clearly a mechanical interface, whether or not we want to call that a mind.
So, getting back to your paper, I’m thinking the human looking at the computer screen and reading the word “cat” is more like a component inside my device than me collecting $20 bills from my kitchen table. That is, the human is part of a larger computing device using a mechanical interface between components.
Or, framed even more broadly, I’m not convinced that computation has any meaning outside the direct physical changes the computer makes to the environment. I say outputs are always structural. That humans believe that they assign interpretive non-structural meaning to outputs is a quirk of our physical implementation.
By the way, I’m not sure if I’m agreeing or disagreeing with you. I’m perhaps talking about a slightly different problem. I still need to read your paper.
(On preview, I see the thread has probably already covered my points, but in any case, here it is.)
Which is not a counterargument, since it’s logical possibility you need, not material possibility. What this shows is that consciousness isn’t (logically!) necessary to generate this output behavior, hence, any realization is consistent both with the option of the device being conscious, and with it not being conscious.
But that’s really not the issue I’m driving at with the input-output behavior. It really comes down to the question: you find a dictionary of words in an unknown language explained in that same unknown language. Do you believe you can translate it?
