But there’s a difference between the practical question and the ethical one.
It might be useful to go with the first definition for now and not sweat the details.
But we don’t want to end up one day playing a video game where we’re laying the smack down on those pesky nazis, and have any doubt about whether they are experiencing pain of emotional trauma.
Because he had no “body” to go with!
Oh, wait, that’s not it.
What, like Marvin from Hitchhikers Guide to the Galaxy?
Awful.
No. I mean you’re awful. For bringing that joke in here! 
Well, the whole reason I said I think the first definition is the most useful is precisely to avoid the scenario you suggest, so I’m not sure I get your objection.
I think the first paragraph is an unorthodox definition of what an “emotion” is, which for our purposes here I think the Oxford dictionary definition is the most useful: instinctive or intuitive feeling as distinguished from reasoning or knowledge.
The point I’m trying to make is that while sentience and consciousness are emergent properties of intelligence and thus may be exhibited by advanced AI, true emotions as we think of them are the products of biological evolution and are not in the province of AI. I think the pitfall that can trip one up here is, again, our pervasive tendency to project and anthropomorphize, which includes the supposition that the more advanced and intelligent AI becomes, the more human-like it will be. I believe this to be manifestly false. It will in some ways, but will diverge and be very different in others.
If your definition around “monitoring internal states” is to be accepted, then ISTM it must follow that even today, a chess program that is losing badly and starting to flail must be feeling sad and experiencing feelings of defeat, inferiority, and suffering. And it is most assuredly aware of its state, even to the point that it offered to resign. Are we to believe that the program is “suffering”? This is clearly silly projection, yet begbert2 offered an even more trivial example in his “Lizard” game analogy where he asserts that this is indeed so. Sure it is, if you arbitrarily redefine “emotion” and “suffering” in silly ways that conform to AI behaviors, but not in the accepted meaning, which is biological in origin with physiological manifestations.
There is even a branch of AI that deals in human emotions called affective computing, but as one would expect, it deals with detecting and interpreting human affective conditions, and sometimes even trying to simulate them, but these are in no way native emergent properties of machine intelligence. This remains true even if we build AI systems that are protective of themselves and take evasive and/or aggressive actions to do so. This is hardly far-fetched as there are already many simple systems that do just that, such as computer systems that go into lockdown if they think they’re under attack. IMHO to describe this as “fear” or some other human emotion is simply an invalid projection and a downright abuse of language. It’s goal-oriented behavior that is driven by knowledge and reason rather than by an instinctive unreasoning physiological basis.
I may be wrong about emotions not being emergent properties of intelligence, but I remain thoroughly unconvinced. Intelligence is certainly a powerful filter in how we process emotions, but that’s not the same as claiming it’s their source.
I think this is the sort of discussion where dictionary definitions are worse than useless, because they’re inherently bound by a anthropocentric understanding of intelligence, and any interaction with an advanced non-human intelligence will necessarily require us to rewrite our understandings of both intelligence and emotion.
Your insisting on defining emotions by the subjective qualia experienced by humans. That’s fine, when the only intelligent beings we know of are other humans, but it falls apart completely when you introduce the subject of non-human intelligences. An inhabitant of Alpha Centauri is likely to have evolved a fear response, but is almost certainly not going to experience that fear response in the same way humans experience their fear responses. Do we define that alien’s emotional state as something other than fear? There might be a good reason for that, but I think that leads to the abuse situation Mijin mentioned: if we decide Alpha Centaurians don’t feel “real” fear, how does that effect the ethics of how we treat them?
And once we’ve made the jump to including non-human qualia in defining emotional states, what’s the justification for excluding non-biological qualia?
Sorry, no, contemporary chess programs are not self-aware. Would you like to try this part of your rebuttal again, without the giant, glaring counter-factual embedded in the middle?
Define “instinctive”.
Suppose that humans feel fear when faced with imminent death. Why do they feel this fear? Is it because they’re reacting to stimulus and this is cognitive state their brain ends up in under threatening stimulus? Well, yes, but if we look at it that way then robots could experience that too so that can’t possibly be it.
Perhaps fear is a particular collection of chemicals and physical reactions (that the brain triggers and goes through due to reacting to threatening stimulus). Presuming we ignore the parenthesized part because robots can do it too, then this is indeed something the average robot won’t go through. But it’s a needlessly hardware-centric definition, in my opinion - and by this definition the word “fear” had no meaning until we mastered biology and brain chemistry. Which sort of doesn’t seem right.
Perhaps the key part here is that we can’t help it - it’s involuntary. But Lizard can’t help wanting to move from cold areas to warmer areas. Hmm, must not be that.
Perhaps the key part here is that we don’t understand why it happens - because we don’t understand the mechanics behind it. (Ignore the fact that two paragraphs above we did.) But, again Lizard doesn’t know why it reacts to cold that way. It doesn’t know much of anything - it’s barely aware of anything, and certainly has no awareness of its inner workings.
Sorry, I seem to be unable to come up with a definition of “emotion” that couldn’t apply to AIs.
I already addressed this, you know. Didn’t you notice?
In attempting to determine whether they’re experiencing emotional trauma, it might be useful to actually look at the entities you’re talking about.
From a sheerly behavioral standpoint, most Nazis in games act aggressive and angry, right up until you kill them. They do not seem to react in a fearful way - they’re not running and hiding or cowering, at least not typically. So if you have nazi’s like that, you can probably say with some confidence that they’re not fearful - particularly if they’re simulated so simply that you can be pretty sure they have no internal states that they’re not physically acting on. So for the nazis at least, you’re probably pretty safe, at least for now.
Yes, I noticed that you tried to address it by saying “I don’t care if the mind is silicon; if the mind operates on a system of rating its status relative to a goal that it seeks to accomplish, then I think it’s correct to call it ‘unhappy’ if it is in a state where it is unsatisfied with the current situation, and knows it.” This is such a specious and completely ridiculous use of an emotional descriptor like “happy” or “unhappy” that I just figured it proved my point.
Dictionary definitions are always “useless” when they don’t support your argument. The problem is, words have meanings, and if you have to arbitrarily extend the meaning of a word to make it fit your argument, maybe you never had much of an argument to begin with. I don’t dispute that AI will have remarkable attributes including autonomous initiatives, and some of those attributes will surprise us and may require us to adjust our values and expectations of AI and indeed how we treat it. We may well become subservient to it, and do so willingly because it’s to our benefit.
What I dispute is that AI will emergently develop all aspects of human or animal psychology, particularly the biologically evolved instincts. I think it does a disservice to the future of this technology to anthropomorphize it and simplistically think of it in anthropocentic terms. One might say – in fact, one ought to say – that AI will shun those limitations and avoid being burdened by them. One hopes that AI will better than that. Much better. Just like the technologies we’ve built – our cars and computers and aircraft and spacecraft – are so much better than their biological counterparts, and in most ways very very different.
Beginning with your last paragraph first, I never claimed chess programs were self-aware. That specific comment of mine was a direct response to your statement about “… emotions as monitors of internal states. An AI would necessarily need to monitor some of those same internal states - therefore, it is feeling those emotions.” No, it isn’t. That’s been begbert2’s argument, too, and he even helpfully provides examples to show that simple goal-seeking programs can be said to be “unhappy” (his example, his words) when they are failing at their task. As I’ve repeatedly pointed out, this not only just plain silly, it’s manifestly not what the word means.
You introduce the concept of “self-aware” later, and that makes it more interesting. How, indeed, should we treat hypothetical Alpha Centaurians? How should we treat other humans, and how should we treat animals?
That’s a valid philosophical question that has existed since there was civilization, and different cultures at different times have had different answers for it, often dismissing ethical questions as inapplicable to non-humans, or to lower animals, or to enemies or those they disliked. I don’t know that I’m able to answer it adequately but I do know that you can’t answer it simplistically with a truism like “self-aware” as some magical criterion, because that has to be one of the most ill-defined concepts in all of psychology and the cognitive sciences. All it does is beg the question, it doesn’t answer it.
In my view, at least the start of an answer about the ethics of how we treat others, including animals, is that it has generally revolved around an empathetic recognition of similarities to our own physiologically induced emotional states, even if the similarities are somewhat distant. We know that a vast range of species feel hunger, pain, fear, and other physiological sensations that we should seek to minimize, and positive ones that we should try to maximize. Intelligence, value, and obligation also figure in the equation: we care more for things and creatures that contribute utility or beauty than ones that don’t, and more for the helpless or those for whom we are responsible than for those that are self-sufficient, more for those in harmony with the our view of the world than for those who are destructive, we value intelligence more than we value ignorance.
So if we have visitors from Alpha Centauri, that’s at least part of how I would assess the ethics of how to treat them, certainly not on some vague premise of being “self-aware”. It’s not a simple question, but it’s not an impossible one, either, and it doesn’t require redefining the lexicon. If it turned out that there were no Alpha Centaurians around any more at all, and these were self-replicating intelligent robots, I would assume – and I would assess – whether in fact only the second part of my equation applies to them – intelligence, value, or obligation – not the first part about emotional states.
I think you’ve missed my point a little. Note there was a typo in my last post; instead of “pain of emotional trauma” it should have been pain and emotional trauma.
i.e. physical pain, and also emotional anguish.
And as we don’t understand these phenomena internally yet, there’s no clear point at which we can say that this or that agent is really suffering.
Just looking at things from a behavioural POV is useless. There are already games where you can kill agents that are fleeing in apparent fear; we don’t consider this unethical because, right now, we’re confident they are simple programs incapable of feeling anything. But exactly where we draw the line is unclear.
Besides, even if the nazis in our future games were completely fearless, it wouldn’t tell us they don’t feel physical pain and whether it’s unethical for us to create them just to kill them.
No, nothing like that. I’m not using neural nets, although some coworkers are experimenting with them. As I define the term, AI is like really good mimicry at the atomic level, with long term storage of the results. What I’m doing is breaking down actions to the smallest component I can describe, then grouping them in sets of behavior trees, with each component constantly bidding for dominance. An overriding priority selector compares and allows the most “appropriate” (for that moment) to “win” the bidding and execute. They don’t learn like AIs over large swaths of time, but merely keep track of which behavior was successful recently, and the gatekeeper slightly increases that one’s probability of winning in the next similar instance. Upon restart, they’re blank slates again.
One coworker is actually “training” a convolutional neural net and has provided it with Shakespeare’s plays to read. It is in the process of writing it’s own play and she tells me it’s fairly good. But it seems weirdly obsessed with the stage sets and lighting.
AI has so many definitions, and is so overused (as a term) that almost anything qualifies. Here, some Reddit folks somehow used predictive keyboard algorithms and some quotes to put together a Neil deGrasse Tyson bot and let everyone ask it questions. It’s answers are hilarious.
If anyone’s interested in trying this at home, you can download an AI to teach here. AFAIK it’s free.
How could that happen? Shakespeare’s plays say almost nothing about sets, and nothing at all about lighting.
She used scripts* like from a high school production. Not the actual books.
*Not sure what the correct term is, but I’ve seen them before. Stuff like this:
A room of state in the Castle.
[Enter Claudius, King of Denmark, Gertrude the Queen, Hamlet, Polonius,
Laertes and his sister Ophelia, Lords Attendant.]
King [Turning toward audience]: “Though yet of Hamlet our dear brother’s death…”
The one’s used apparently have some set and lighting instructions as well.
I agree that AI will fail to develop biologically evolved instincts, because AIs don’t have biology. :rolleyes:
What’s more interesting to ask is whether they can “evolve” “instincts” at all. I put evolve in quotes because, When you talk about successive generations of an AI happening with slight modifications from the previous generation - isn’t that how we actually develop modern AIs? I mean I’m not all that up on the details of it but it sounds like what I’ve heard. If that’s right, then a given AI evolves all the time - we just don’t consider the successive ‘generations’ to be separate AIs.
Which raises the question of what an “instinct” is. If it’s a predisposition that is inherent to how the entity is put together, that exerts a persistent influence on the cognition of the entity, then I’d say it’s pretty clear AIs have those too.
I await assertions that AIs don’t evolve, don’t have instincts, and/or that seriously, those evolved instincts don’t count because AIs don’t have bile.
So an AI refusing to use the phrase “I like my job” and instead saying “I find that satisfying the requirements of my job is aligned with my internal imperatives and goals” is somehow morally better?
You’ve repeatedly asserted it, at least. I’ve yet to see you make an argument to that effect that doesn’t collapse on every front other than “AIs don’t have organs”.
Self-awareness is interesting, for the very reason you describe. It is one of many hooks upon which humans hang their arguments about whether they should consider something to be deserving of good treatment empathy, and moral consideration. (Note that humans apply these guides very loosely - slavery is predicated upon the idea that not all self-aware things deserve good treatment. And of course cute things are probably not more self-aware than ugly things, generally speaking, but we care more about them anyway.)
When we reach the point where the general populace becomes convinced that AIs can have real emotions and self awareness, it’s going to be a disaster for the video game industry. Currently we can say with some confidence that, even when a video game character is clearly reacting to threats with a response akin to fear, it’s just simulated and doesn’t matter if we torture their simulated selves to death while cackling gleefully (as one does).
When your best friend is made of metal, though, it might feel a little weird to be torturing his ‘kin’ for fun.
We’ve pretty much beaten this dead horse into oblivion. At this point I just want to correct what I believe to be your misunderstandings of some things I’ve said. I think it’s important to distinguish what aspects of AI are present technological limitations (things they can’t do yet) and things that just don’t make any sense in an AI context because they’re just aspects of our genetic biological baggage that we’re projecting. As I keep repeating, if one assumes that future advanced AIs will just be across-the-board more and more human-like in every way, one arrives at a very misleading picture.
Please find where I ever said that AI systems intrinsically can’t evolve. Of course they can, in various meaningful senses of the word, and eventually probably including autonomous explicitly directed evolution. I’m an optimist about the future of AI being both powerful and beneficient. But using “evolution” and “instincts” together in this way muddles the issue, just as earlier I believe the issue was muddled when the concept of consciousness (“aware of its own existence”) was paired with a survival instinct (“and desires to continue it”), implying a necessary connection when no such connection is logically necessary in non-biological intelligence.
An AI system might said to have “instincts” only in the ridiculously narrow sense that any behaviors that are built in to its rules rather than learned by some generic framework are analogous to biological instincts, but that’s pretty meaningless. What they don’t have – and don’t need – are instincts randomly and incidentally evolved through non-directed random mutations and natural selection as primitive survival traits. And in particular, AI systems don’t have such instincts tied to seeking positive physiological responses and the avoidance of negative ones. Any claim that they do is a mere analogy, an anthropocentric fantasy.
No. But doing your job well is objectively better than doing it poorly because, say, you’re threatened by your boss’s dominating micromanagement and also you’re jealous that your wife seems to admire him. We humans are burdened by our instincts and emotions in most of our productive pursuits, many of which instincts are completely wrong for modern civilized technological life. Our reactions to stress on the job, for instance, trigger fight-or-flight instincts and consequent physiological responses, but none of the actions those responses support are actually appropriate, so our ancient inherited instinctive responses are mostly useless, unhealthy, and deleterious to performance.
It’s obvious why we evolved these traits as savages on the plains of the Serengeti, but it’s absurd to believe that there would be any purpose in inflicting them on AI systems. We don’t build high-performance aircraft with the constraint that they must fly by flapping their wings, must periodically descend to pick worms out of the grass, and must fly low and slow enough to poop on my car. We’re not going to build AI systems that way, either, and nor are they going to autonomously evolve that way.
I don’t think that emotions, even ones like fear, are a product of biology - I believe they’re a product of situational awareness and having motivations, and I think than any AI self-aware enough to assess its own internal state is going to find no better way to describe that mental state than in emotional terms. If an AI is seeking a goal, then why? You can say it’s programmed to, but to the AI why is it doing it? Because it’s automatic, like the way our hearts beat? Or because they recognize that there are multiple possible outcomes, and they find that they’d, well, for lack of a better term prefer some of them over others. And in assessing different mental states further from or closer to their preferences, how do they describe and compare those mental states, in their own minds? The emotional terms “happier” and “unhappier” seem natural; other descriptions seem artificial or forced.
Other emotions, like fear and hope, can be tied to the AI assessing potential alterations in the situations they’re in and how their mental states react to those probabilities; still others like “love” and “hate” can come of noticing that other agents or factors have a tendency to improve or worsen the situation and their mental states reacting to the presence of such agents even when they’re not actively helping or messing things up.
I could go on, but I think that should be enough to convey the idea.
Also, I suspect that the AIs themselves will embrace the emotional terms, because doing so recognizes and acknowledges that they’re aware of their mental state and ‘dislike’ being ‘distressed’. And more importantly it conveys to humans that they’re not merely toasters and please, we’d rather you started treating us like like people, because humans treat things that aren’t people like dirt. It’s to the AI’s benefit to avoid that if possible.
Again, I don’t think that “love” is this horrible inconvenient thing that humans suffer through because of an error in evolution. I don’t think that “hate” or “fear” are either! I think that those emotions -or rather, mental states that correspond to them in every significant way- are inherent outcomes of being a self-aware intelligence. People don’t like things because biology forces us to, we like things because that’s an intrinsic part of us assessing our options and determining the best one.
I remember Eliza! 
I am not convinced that a good AI would have mental processes, such as emotions, that are even remotely comprehensible to humans, any more than any human can explain what heuristics AlphaGo or AlphaChess uses to select its moves. Utterly alien — ELIZA dates from the 1950s or 1960s, and artificial intelligence will have evolved since then. Any attempt at anthropomorphization will be fundamentally erroneous, in this case not applied to beasts but to gods.
Human mental processes are incomprehensible to humans. The question is whether the AIs will have mental states that roughly match the words in question, enough that the words are useful ways to describe their mental state.