Oh – I just remembered another person that might be of interest to you DSeid – Tom Ziemke, who is advising someone on tools to analyze neural nets. Once again, I have to claim substantial ignorance as to the status of his research.
SentientMeat – I assume that would be Laird of the SOAR cognitive architecture? Do you have any opinions on SOAR?
Well since we are overloading each others reading lists …
Gardenfors’ book is Conceptual Spaces: The Geometry of Thought. In it he does an excellent job of presenting concepts as geometric objects and brushes up against my thought of geometric transformations of those objects when he discussed the color spindle and its relationship to what we call various skin colors. Turns out that the names we use for skin colors (White, Black, Red, etc) map nicely relative to each other onto a color spindle reduced and shifted (translated) within the domain of the larger color spindle. I see this as a model for creativity in the Hofstader version of creativity as metaphor making. (As in his book Fluid Concepts and Creative Analogies: Computer Models of the Fundamental Mechanisms of Thought.) Say along the lines of having an conceptual object of a wheel and applying that shape (tranforming/translating it) into the different domain of thinking about the planets and the stars. Having the conceptual object of a solar system and translating onto the domain of considering atomic structure and creating the Rutherford model … and so on. I further consider that the edges of these objects are somewhat “soft” to various degrees, probably in similar magnitude to what Grossberg would refer to as the “vigilance” for each potential matching process. I’d love to have the skill set to take this to the next level and integrate Gardenfors work with Grossberg’s and with my image of geometric transformations/rotations/translations … but it is, regrettably, one of the many areas where my desires outpace my knowledge set.
If you want a concise review of Grossberg’s state of the art do look at that autism article I co-authored with him. Its linked to on that same page, at the top. In it he was forced to review most of ART, CogEM, and spectrally timed ART (START) fairly concisely, yet clearly enough that a clinical audience could absorb it, in order for us to apply it to the problem of understanding autism. You can skip over the first half in which I review autism and our joint application of his models to that particular problem.
Well, I asked, after all. 
And I’d like apologize to Daniel for sidetracking this thread yet again. But I don’t feel too bad about it, as I figure you’re OK with it. Nonetheless, I’ll be brief.
Serendipity! As it turns out, I’ve been carrying around a paper (first one on Deb Roy’s publications page; he works on Cog or Kismet at MIT) to read whenever I had some down time. A mate o’ mine defended his PhD today, which gave me a chance to start it while waiting for the presentation to start. It turns out that Gardenfors is one of the first models he (Roy, that is) mentions; if I hadn’t been sensitized to the name, I wouldn’t have noted it. (It looks like there’s yet another body of work with which I’m very much unfamiliar.) If you’re looking for related work, it seems like Roy’s survey is a good place to start.
From reading that first Grossberg paper you cited, I like ART. I don’t fully grasp it (yet), but it looks like it fills some vital role(s). Interesting that he laid some groundwork for Kohonen nets – I didn’t know that. On your recommendation, I’ll read that review; I hope it clarifies things for me. Are you familiar with (can you reference) any criticisms of ART?
Jeez, Louise. 85 pages? From Psychological Review journal? You, sir, are a cruel, cruel, man. 
If personality is 90% to 100% genetic, then how do we end up with such a variety of cultures?
Do the Japanese not learn how they are to act in accordance with the Japanese culture? And isn’t this very distinct from that of the US?
While I am a strong believer in the genetic component of personalities, especially after having kids and seeing the variety from the moment they were born, I also believe there is a learned component. Given the amount of learning we do, how could our environment not make a substantial difference?
DSeid, it’s occuring to me that you are making a distinction between personality and values, to the point of calling personality the thing you get from genes, only, and values being stuff you learn, only.
I’m not aware that this is an accepted distinction, is it? If so, then the point in Ray’s book is not a point at all, it’s by definition that way, which means it’s hardly worth printing.
I meant Pinker’s book.
Dig, why do you think I told you to just skip all the autism specific stuff? Seriously, one of our concerns with this article has been that the autism background reading will be too much for the modelling people to wade through and that the modelling background will be too much for those on the clinical side to handle or at least stick through, so that no one will actually end up reading it. We’ll see what actually happens. I am not familiar with any critics of Grossberg’s stuff.
btw I bought “Wider Than the Sky” yesterday. It’s in the queue.
RaftPeople, no that is not what I am trying to call personality vs values. Values are what we consider right and wrong and what we think is important. Personality is our essential traits, like adaptability/rigidity, introversion/extroversion, emotional lability, etc. Personality is what we use as our vehicle in pursuit of the goals we have based on our values.
As to cultural influences on personality, I am sure that it occurs but less so than it does for values. Stereotypes about a national character type seem to be less well founded than people think. See this Science article.
(From the actual article , the link is to the abstract.)
Dig and DSeid - you two are far more erudite in these matters than I. I’m really just an intereted layperson. My opinion is only that the mind will be explained by some computational model, without really being able to comment on any particular aspect with any authority. Still do keep going, and I’ll try to chip in occasionally.
Besides DSeid’s response, I think it’s relevant to point out that Pinker’s 10% figure is meant to express parental (or, “family environment”, as Daniel quotes) influence. Pinker’s claim is that the genetic contribution to adult personality is in the 50% range. Or perhaps I misunderstand your point, as you explicitly acknowledge the role of genetics.
I’d like to qualify my outburst, as I was considering it while making my morning coffee. Being in CS, most of the papers I read (and write) are on the order of 6-8 pages. Even so, they’re really dense. Most journal papers end up being between 15-25 pages. I opened it up hoping to be able to skim it before bed, saw the 85 pages, and kinda wigged. It’ll have to go on the queue to receive the attention it deserves; I have a journal paper to finish that should’ve been out a couple weeks ago (the move from gcc 3.x to 4.0 is totally screwing me).
Well, thank you for the vote of confidence. However, in re-reading the thread last night, I realized that too often I approach this topic exclusively from my CS perspective. That is, I tend to focus on the low-level, “how would I implement this?” viewpoint, missing the high-level issues (a product of being a programmer prior to an AI researcher). For instance, I’m not sure I treated LHoD’s points on evolution from the appropriate perspective.
In many cases, it seems to me that this is good – for instance, I think a huge problem with many philosophy discussions is that they fall victim to an overapplication of Occam’s razor. An example is the causal powers of mental states, usually of the form “Mental state M maps onto brain states B1, B2, B3”. A problem with this is that states are treated as static entities, in which the dynamic process is ignored. Removed from implementation considerations, a crucial aspect gets lost. It may very well be that brain states B4, B5, and B6 are necessary for facilitating the formation and function of M, even though they seem to be incidental.
With that said, I have to be careful to recognize and acknowledge this bias. It’s invaluable to me to have other viewpoints, particularly those that do not come from a computer science perspective. Not to mention the exposure to things outside my everyday interests and the fascinating discussions that result.
[Dr. McCoy]Dammit Jim, I’m a doctor not an AI theorist![/Bones]
Really SM I doubt that I am any more erudite than you are. What happened with Grossberg is more a tribute to his open-mindedness that anything else. I had just been googling about (actually looking for some AI work on humor, thinking of it as a high level figure-ground reversal process, realizing that such required parallel processing streams working nonlinearly) and stumbled into his website. I found his articles fascinating, albeit hard to comprehend the first time through, and ended up e-mailing him some questions. He was gracious enough to answer my questions and engage in an ongoing give and take. I have a professional and intellectual interest in autism and was very familar with the then new work linking autism with cerebellar deficits, and became awed by how well plugging that into my understanding of his models resulted in much of autistic phenomenology. I wondered if was considering doing an article applying his model to autism and encouraged him to do so and he responded by saying that he didn’t currently have the time but that he would be willing to comment on something that I wrote. From that emerged a several year process of collaborating along with multiple revisions. We actually only met more than a year into the process! Nah, I’m just a pediatrician with a pathologic case of intellectual curiousity and a big dose of chutzpah who was fortunate to come into contact with an intellectual giant who was open to unsolicited questions. One thing I learned in the process is that I am glad I chose a clinical rather than an acadamic life! Dig is the expert opinion here.
Speaking of whom … Dig I really only suggested that article because the review of his models is quite good in it. It was written with the understanding that some readers would be from the clinical side, so clarity was essential, but it was also written knowing that our application of his models required integrating ART, START, and CogEM in a real world dynamic process in which forces become imbalanced, so the explanations had to be complete enough to allow for that. That background section should be a relatively quick read for someone who comes to it with your knowledge set. Now the autistism bits may be a bit harder on you! And seriously, no offence taken if you never read them and even less if you take a long time to do it. 
I’m still reading and still interested, but a lot of this is over my head. Don’t feel bad about hijacking; it’s very educational!
Daniel
DSeid, thanks for clarification on the personality vs values and percentages. I had read the thread one day and remembered the 0-10% and hastily popped in and posted later on without really remembering the entire OP, so basically I forgot that he had allowed for signifcant “other” influences.
I agree.
At the risk of sounding like a hobbyist that simplifies things a little too much (which I am), I think Pinker and Chomsky and others seem to be missing the elegance of a connectionist model. The structure is innate, which provides the general capability to abstract not just single words/ideas but groups of words/ideas. The actual combinations of valid types of words, phrases, sentences, etc. are learned.
This is exactly the strength of connectionism (matching unfamiliar inputs to previously experienced inputs) and eliminates the need for some super grammar, etc.
I don’t think the computational model is worthless, but given the success with the connectionist model on duplicating some basic functions of the human brain (visual recognition, speech recognition), and given, what I would consider the lack of success with the computational model (are there any successes?), I don’t understand how anyone could ignore the connectionist model.
Why can’t we get all of our mileage out of reactive systems?
Why does “mind” need to be anything more than a brain that not only reacts, but also incorporates itself into it’s own world view for better predictive power?
Whether a simple or very complex brain, I don’t see the need for anything beyond reactive, whether that is to external senses only or to a combination of external and internal state influenced by genetically provided structures (e.g. temporal mechanisms).
I am only able to pop in extremely briefly but allow me to chime in that Pinker would very much object to being grouped with Chomsky. In Words and Rules (which was, btw, a great book) he provides critiques as damning of Chomsky as he does of strict connectionism. Hs actual position is much more nuanced and researched and evidence based than any of those dogmatic stances. I actually read his proposals as being very similar in structure to what ART proposes just specific to linguistics and without the mathematical modelling and consequent generalizability. I will quote from that long article I’ve referenced an introductary summary of ART;
The problems with strict connectionism are many. Most of all there is no evidence that real systems operate in that way at all. Strict connectionist system require a teacher and are prone to catastrophic forgetting. Biologic systems have the ability to prime from the top-down to various degrees depending on the circumstance and the system. There are some top-down prototypes that we are wired to start with, as well as ones that are learned, and clearly these inate ones include some basic structures of language and the ability to recognize the exceptions to the rules.
The brain is not a sequential processor yet it does function with modules, just in a very nonlinear manner. The two concepts are not mutually incompatible.
There’re a couple issues here, which I’ll get to in a moment.
You might want to go back to my post #16, where I quote from the computationalism article linked to by SentientMeat (in fact, it’s chock full o’ good information). Computationalism provides a bridge between syntax and semantics. And unless you’re prepared to deny that humans use symbols in their thinking, the computational model provides the only model thus far that accounts for that (to my knowledge). I’d also like to echo your own words back at you: “success with the connectionist model on duplicating some basic functions”. That’s it in a nutshell, from what I know. Sure, neural nets have been designed to do some speech processing, etc. By the same token, one might consider SHRDLU or Flakey the robot resounding successes for symbolic computation. (Which they were; I’m not trivializing them.) But it’s not enough; we don’t know how to get symbols from connectionist models, much less, as I said to LHoD above, even have a good idea of how to analyze the networks.
Which leads me to address the issues I alluded to first. In particular, assuming I’m reading your use of the term “structure” correctly, the structure of neural nets, at this point in time, are rigid, brittle, and must be divined through trial and error. Which is why I’m so enamoured of “growing neural nets”, particularly of the hierarchical variety (paper by Rauber cited above). And not only that, but the networks we have are relatively simple and serve only one purpose. (Disclaimer: “neural networks” is a huge field with which I’m not heavily involved.) Now, the Roy paper I cited above gives some indication that work has been done in mult-modal networks. But I get the impression that they use a single network, simply expanding the number of inputs. Not only does this obviously not match actual brain development, but it also violates the essence of MMM (assuming you accept that).
The fact that you can make long term plans says that there’s more to brain function than a purely reactive system. As you say, with “temporal mechanisms” (what would they be like? how would they work?), perhaps it would be adequate. But, in general, that’s not what “reactive” is taken to mean – we need to distinguish finer levels of function. This is not to say that you’re wrong, just that you and I may be speaking in different terms.
So, in the literature, a distinction is made between reactive, deliberative, and reflective “layers”, where “reactive” refers to tightly coupled sensor/effector connections or immediate stimulus/response mechanisms (e.g., motor control), “deliberative” refers to planning capabilites (e.g., “what path should I take to move from point A to point B?”, in addition to “how do I handle situations where that plan fails?”), and “reflective” refers to the ability of an agent (robot, human, software program) to observe and reason about it’s own states (e.g., “I don’t feel good, so I’ll go to the doctor” is reflecting on internal health state; “I have to be in court in an hour, so I’ll go to the doctor after that” is reflecting on a deliberation).
So, to tie this back into the above (and the original text from which you quoted), this is an issue because reactive systems (such as the basic functions we’ve developed in neural networks) do not do the work that we need them to. How does one get a symbolic plan (e.g., to get to the doctor I have to get in my car, go out my driveway and make a left, etc.) from a neural network? We just don’t know (yet).
Thank you for elaborating on that; I had a feeling that my impressions were a tad off.
Can you tell me what you mean by “strict connectionism”? I think I get what you’re saying, as you imply that ART doesn’t fall in that category. But there are lots of connectionist systems that don’t require a teacher, so I’m unsure as to where the boundary lies.
I’m occasionally looking at that autism paper – obviously, ART has some tremendous benefits, but I’m not clear on how it works in detail. It’s a bit early for me to ask these questions, but I can’t resist, due to what you’ve just said. Does ART actually make use of multiple modules? Are these modules pre-designed? Has any work been done on dynamically changing the structure of the network? Just throwing those out there in case you can answer them…
My understanding of the position of anyone in this field is limited to the googling I’ve done after seeing their name posted, but Pinker and Chomsky certainly are lumped together in numerous places.
Not that I’m disputing what you are saying, just that it’s not an unreasonable conclusion based on what is out there on the web.
Wouldn’t you say we have evidence based on the information that biologists and neuro-scientists have accumulated, coupled with the ability to achieve similar results with ANN’s for certain tasks?
Humans have 2 techers:
If this is true, it doesn’t mean connectionism is the wrong model, it could mean we have not learned how to create systems that avoid this problem.
Is there evidence that we recognize exceptions to the rule in language prior to learning the language? It seems logical to me that we have the ability to process language, but that we learn the specifics.
I agree, I think that the brain breaks down processing into what you might call modules, and feeds the output from one module to the input of other modules. But I think it would be a mistake to think that those modules are “clean” and independent.
Sorry for picking up the post to DSeid, but I was just popping in and thought I could answer sufficiently.
I don’t think anyone disputes that the brain is a neural network; the qualification of “strict connectionist” models is important. I think DSeid was specifically talking about backpropagation networks, which are distinctly unlike biological nets.
I think there might be a disconnect in terminology here; when talking about “teachers” in the context of neural nets, it implies what is called “supervised learning”. For instance, in backpropagation nets, there is a training set of data that is fed as input to the network (usually repeatedly, in a randomized order) for which there is a known output (or answer). The weights of the network connections are adjusted according to the error value of the network’s output as compared to the input instance of the training set. I forget who it was (Widrow, I think?), but a huge leap was made when a formula for attributing credit/penalty to earlier layers of the network (that is, mathematically determining how much to adjust each connection weight, backpropagated from output nodes to the input nodes). I think that ART does something like this also, using “patterns” from memory as the output. I’m leery of making that claim, as Grossberg says that ART is based on similar principles to Kohonen networks, which are unsupervised.
I don’t think that’s what was implied; rather, “strict connectionist” models are prone to catastrophic forgetting. ART, on the other hand, are not.
I’m not gonna really touch this one – if you’re interested, Elman created a recurrent network model (which bears his name) that proved that a network could indeed learn grammar rules, including “exceptions to the rule”. If I recall correctly, there was quite a row between the connectionists and others (Levesque in particular, I think…it’s been a long time) about whether connectionist models could meet the language challenge, or whether the Chomskian idea of innate, universal grammar was correct.
Also not touching this one, as it’s rather contentious. However, I pretty much agree – I think connectionist models are up to the task.
Well “connectionism” is usually used (and this includes its use in Pinker’s books) to refer to what I understand to be “back-prop” models (eg McClelland’s work) and usually to systems that make little or no attempt to model what we know about actual brain function. I may be biased by my admiration of Grossberg’s models and his assesment of back-prop:
(I wish that my copy of Words and Rules wasn’t packed away, but we are trying to sell our house and it got stored in the “decluttering” effort. His critiques are quite good as I recall.) As someone obviously enamoured of Grossberg’s work, I am frustrated that his models do not get as much wide play as I think that they deserve. He does not however write for a general audience as a general rule.
Boy, you will like that paper I think. Here is the continuation of the intro to the theoretical background section:
Output needs to be adaptively timed to deliver results. This is accomplished through adaptive timing using spectrums of outputs. This he calls spectrally timed ART or START. As the paper continues:
And the point of the application of his models to autism is that it shows what happens dynamically when these functions become imbalanced during early development. This then becomes imbalancedSTART or iSTART.
Raft, I tend to agree with much of what you said.
I will go back and read the articles, I’ve had limited time so I’ve been charging forward because I enjoy the topic (in my spare time for the last couple years I’ve been working on an alife/ann/ga project so I think about and analyze this stuff quite a bit, I just haven’t read up on the published papers).
Side note:
Ok, I’m reading the links from Sentient’s post and I see this in the “Weaknesses of NN’s” department:
“For example, connectionists usually do not attempt to explicitly model the variety of different kinds of brain neurons, nor the effects of neurotransmitters and hormones”
This is exactly the type of thing I set out to do in my project. My thought was twofold:
All depends on your definition of “symbol”. I read the computational link (quickly) and could not find a satisfactory definition of “symbol”. If it means “some abstract notion in the brain that represents either a set of other abstract notions in the brain, or it represents some physical entity”, then I would say that yes we use symbols, sometimes, for some things.
I agree that SHRDLU and Flakey seem like successes (I just googled them). But for me, I feel that one of the most important aspects to human intelligence is it’s flexibility and adaptibility. I’ve been programming for a long time and what bothers me about any system like those you listed is the “brittle” nature of software, all paths must be considered in advance.
Also, a slightly different thought is that it’s possible our brain is emulating the type of processing that is programmed into SHRDLU, in which case it might be more efficient to go straight to LISP, but you might lose the benefits of an NN. Maybe a combination is the best of all worlds.
Valid points. Some stumbling blocks in my project are:
Ok, I was thinking the term “reactive” covered all three layers you mention. You seem to be saying that an NN could not handle all 3 layers, but to me it seems like it could.