# What would a sentient robot/AI really want out of life? **URL:** \ **Category:** Great Debates\ **Created:** [November 19, 2006, 11:39pm UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081 "2006-11-19T23:39:15Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![BrainGlutton](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@BrainGlutton](https://boards.straightdope.com/u/BrainGlutton)\ **Post date:** [November 19, 2006, 11:39pm UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/1 "2006-11-19T23:39:15Z")
SF has often dealt with the idea that a machine (or an artificial organism like Frankenstein’s monster) can develop full self-consciousness or sentience, can become a true [“strong” artificial intelligence](http://en.wikipedia.org/wiki/Strong_AI). And it is always assumed (though rarely explained or defended)\* that sentience necessarily includes independent volition and desire --the ability to **want** something. That’s what, more than anything else, distinguishes Data from the Enterprise’s ship computer – the computer does only what it’s told, but Data has independent free will and desires. But what does a sentient AI want? There are two hoary conventions: 1. The AI wants to become fully human, or as much like humans as possible. Frankenstein’s monster, Data, David in _A.I.,_ Andrew in _Bicentennial Man,_ Cryten in _Red Dwarf._ (Also Pinocchio and the Tin Woodsman.) 2. The AI wants to destroy humans and have the world to itself. The Cylons in _Battlestar Galactica,_ SkyNet in the _Terminator_ movies. But what would an AI **really** want? Either of these, or something entirely different? Or is it possible for an AI to become self-aware by any reasonable standard and still not have desires of any kind, no more than this PC has? Would it simply be a dutiful slave by nature and aspire to nothing else? (Most, but not all, of Asimov’s positronic-brain robots seem to fit this description.) \*In Heinlein’s _Friday,_ the title character briefly muses that an entity becoming fully intelligent necessarily means it starts to ask, “What’s in it for me?” But she simply infers this from observed behavior of some quasi-sentient AP’s (genetically engineered persons/organisms), and does not follow up on this line of thought. ---
**Author:** ![Revenant\_Threshold](https://avatars.discourse-cdn.com/v4/letter/r/f1d935/32.png) [@Revenant\_Threshold](https://boards.straightdope.com/u/Revenant_Threshold)\ **Post date:** [November 19, 2006, 11:55pm UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/2 "2006-11-19T23:55:11Z")
Nitpick; it’s **K** ryten. I think a simple A.I. would want whatever we programmed it to want out of life (assuming we program it to do such a thing, and not build hovercars). I think if we get to a point where A.I constructs can learn for themselves - that is, can add to their own programming based upon the old programming - that while they’ll still be subject to the fundamental rules in their system, it’ll \*appear \* as though they aren’t. So they’d gradually build up new wants and desires from adverts, thinking, reading, just as we do. This is the clincher for the OP’s question, really; at what point can an A.I. programmed just to “drive train from station to station” learn “fewer people usually at one stop, so close the doors sooner to save time”? I think it’ll be then that the illusion of wants and desires will start coming about. I think the final step of creating A.I.s before we have to accord them the full rights that everyone else does is the point at which they’re capable of overriding their base programming permanently; when what they want out of life is entirely due to outside influences and their own thoughts. Which, really, is more than we’d have, because we’ve got innate behaviours still. But I doubt A.I.s would be granted the same rights until they’re actually obviously superior to us, because humans are bad at things like that. (Bear in mind that because I don’t believe in free will, I think the illusion of wants and desires is all we can have; i’m not intending it to mean that an A.I. could never be as “good” as a human). ---
**Author:** ![GIGObuster](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/gigobuster/32/421_2.png) [@GIGObuster](https://boards.straightdope.com/u/GIGObuster)\ **Post date:** [November 20, 2006, 3:24am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/3 "2006-11-20T03:24:24Z")
I’m dealing with this in my NanoWrimo and I have little time, but I do strongly think that there will be a long period were a sentient AI will depend on us, becoming like us will be just a pragmatic way of interfacing with us for the forcible future. What will happen after that? Possibly we will see the writing on the wall and first we will enhance our biological capabilities, until we reach a point when our descendants will find that to get to stars and avoid the eventual end of this planet that we will find a way to give our electronic descendants our [del]programming[/del] knowledge and sence of progress. ---
**Author:** ![BrainGlutton](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@BrainGlutton](https://boards.straightdope.com/u/BrainGlutton)\ **Post date:** [November 20, 2006, 3:31am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/4 "2006-11-20T03:31:18Z")
[QUOTE=GIGObuster] I’m dealing with this in my NanoWrimo . . . [/QUOTE] 😕 ---
**Author:** ![GIGObuster](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/gigobuster/32/421_2.png) [@GIGObuster](https://boards.straightdope.com/u/GIGObuster)\ **Post date:** [November 20, 2006, 3:36am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/5 "2006-11-20T03:36:15Z")
[QUOTE=GIGObuster] the forcible future. [/QUOTE] **Foreseeable**! But speaking of forcible, I agree with Asimov: “He’s a machine - made so.” Even when achieving self awareness, there is the reality (sad or happy depending on your POV) that no matter how advanced it gets we will still be able to pull the plug, or change their emotions at will. So I think we will be not be seeing Colossus, but we will see many Marvins. ---
**Author:** ![GIGObuster](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/gigobuster/32/421_2.png) [@GIGObuster](https://boards.straightdope.com/u/GIGObuster)\ **Post date:** [November 20, 2006, 3:37am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/6 "2006-11-20T03:37:19Z")
[QUOTE=BrainGlutton] 😕 [/QUOTE] [http://www.nanowrimo.org/](http://www.nanowrimo.org/)🙂 ---
**Author:** ![ultrafilter](https://avatars.discourse-cdn.com/v4/letter/u/3d9bf3/32.png) [@ultrafilter](https://boards.straightdope.com/u/ultrafilter)\ **Post date:** [November 20, 2006, 3:41am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/7 "2006-11-20T03:41:57Z")
It’s too hard to tell. Human thought and intelligence is intimately tied to our animal instincts, which are the product of our evolutionary history. AI won’t have any of that, and will develop its thought differently. It may or may not be intelligible to us. ---
**Author:** ![BrainGlutton](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@BrainGlutton](https://boards.straightdope.com/u/BrainGlutton)\ **Post date:** [November 20, 2006, 3:45am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/8 "2006-11-20T03:45:03Z")
[QUOTE=GIGObuster] Even when achieving self awareness, there is the reality (sad or happy depending on your POV) that no matter how advanced it gets we will still be able to pull the plug, or change their emotions at will. [/QUOTE] Go tell the Cylons. ---
**Author:** ![GIGObuster](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/gigobuster/32/421_2.png) [@GIGObuster](https://boards.straightdope.com/u/GIGObuster)\ **Post date:** [November 20, 2006, 3:58am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/9 "2006-11-20T03:58:42Z")
[QUOTE=BrainGlutton] Go tell the Cylons. [/QUOTE] That is fiction. 😉 Cylons will happen if you let go the control of manufacturing capabilities to the machines. Once again, IMO Asimov was correct in the assumption that humanity will put a big stop to that. Even in the SDMB I do remember about half of the dopers got scared by seeing the Asimo in action. Knowing how the average human will react to even a whiff of independent thought close to critical projects, it is clear to me AIs will not be allowed to roam without safeguards. They will have a harder time than Robocop\* had. \*Not an AI I know, but I do think even enhanced humans will have programmed restrictions too, Humans will see to that. ---
**Author:** ![BrainGlutton](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@BrainGlutton](https://boards.straightdope.com/u/BrainGlutton)\ **Post date:** [November 20, 2006, 4:11am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/10 "2006-11-20T04:11:51Z")
[QUOTE=GIGObuster] Cylons will happen if you let go the control of manufacturing capabilities to the machines. [/QUOTE] Which **will** happen, sooner or later, if that turns out to be **cheaper,** despite all voices of reason raised to the contrary. Considering that the manufacturing process will be ultimately under the control of corporate executives who see their chief duty as being to their stockholders. ---
**Author:** ![GIGObuster](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/gigobuster/32/421_2.png) [@GIGObuster](https://boards.straightdope.com/u/GIGObuster)\ **Post date:** [November 20, 2006, 4:18am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/11 "2006-11-20T04:18:38Z")
[QUOTE=BrainGlutton] Which **will** happen, sooner or later, if that turns out to be **cheaper,** despite all voices of reason raised to the contrary. Considering that the manufacturing process will be ultimately under the control of corporate executives who see their chief duty as being to their stockholders. [/QUOTE] Not \*\*fast \*\* enough, anyone doing that will have a fate worse than Ken Lay. And I think you are overestimating and confusing the physical capabilities the AI will have with their intellectual ones. “The fancier ye make th’ plumbin’, the easier it is t’ clog th’ drain.” - Scotty. ---
**Author:** ![Alien](https://avatars.discourse-cdn.com/v4/letter/a/57b2e6/32.png) [@Alien](https://boards.straightdope.com/u/Alien)\ **Post date:** [November 20, 2006, 4:27am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/12 "2006-11-20T04:27:12Z")
[QUOTE=Revenant Threshold] Nitpick; it’s **K** ryten. I think a simple A.I. would want whatever we programmed it to want out of life (assuming we program it to do such a thing, and not build hovercars). I think if we get to a point where A.I constructs can learn for themselves - that is, can add to their own programming based upon the old programming - that while they’ll still be subject to the fundamental rules in their system, it’ll \*appear \* as though they aren’t. So they’d gradually build up new wants and desires from adverts, thinking, reading, just as we do. This is the clincher for the OP’s question, really; at what point can an A.I. programmed just to “drive train from station to station” learn “fewer people usually at one stop, so close the doors sooner to save time”? I think it’ll be then that the illusion of wants and desires will start coming about. [/QUOTE] We got that today already. But self awareness, or desires, are linked to feelings. We want what makes us feel good, and want to stay away from what makes us feel bad. And to a large extent this is tied to our senses. So, can we program feelings? Can we create an A.I. who invents a new God and believes in it? I think I’m with **ultrafilter** on this one. ---
**Author:** ![GIGObuster](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/gigobuster/32/421_2.png) [@GIGObuster](https://boards.straightdope.com/u/GIGObuster)\ **Post date:** [November 20, 2006, 4:30am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/13 "2006-11-20T04:30:57Z")
Expanding a little: the out of control AI makes for great fiction, but the fears one wants to apply to it are like the Y2K bug for computers or the Grey goo in Nanotechnology, the dangers are way overestimated. I do think a more accurate way for sci-fi writers to deal with AI is to concentrate on the frustrations they will encounter when dealing with humans. ---
**Author:** ![GIGObuster](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/gigobuster/32/421_2.png) [@GIGObuster](https://boards.straightdope.com/u/GIGObuster)\ **Post date:** [November 20, 2006, 4:56am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/14 "2006-11-20T04:56:42Z")
[QUOTE=Alien] And to a large extent this is tied to our senses. So, can we program feelings? Can we create an A.I. who invents a new God and believes in it? [/QUOTE] I will once again give away a plot point of mine (I figure this is better than copyrighthing yourself with a self sealed envelope, you have a record with witnesses 🙂 ) IMHO we are forgetting what AI can give to us in the field of medicine: simulations, enhancements, artificial limbs, etc. The need to integrate those elements will lead us to give an AI a logical reason to take care and be aware of its components. I grant you the sensations detected would not be called pain, or hunger. But the AI will learn what make humans tick. As for creating a god, we are going back to a very similar question I made on my first thread created, because it is we who will program it, I see no way to avoid an AI that will be programmed by the fundamentalists of today, the question will be if they will be effective in twisting the logic of the AI forever, because, I have to be brutal here: A logical AI will have little choice but to refuse religion. ---
**Author:** ![BrainGlutton](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@BrainGlutton](https://boards.straightdope.com/u/BrainGlutton)\ **Post date:** [November 20, 2006, 5:08am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/15 "2006-11-20T05:08:30Z")
[QUOTE=GIGObuster] Expanding a little: the out of control AI makes for great fiction, but the fears one wants to apply to it are like the Y2K bug for computers or the Grey goo in Nanotechnology, the dangers are way overestimated. [/QUOTE] What makes you think the “gray goo” danger is overestimated? If we can develop nanotech to the point where it has any beneficial uses, surely someone can develop it as a weapon . . . and one as difficult to control, once released, as biological weapons. ---
**Author:** ![GIGObuster](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/gigobuster/32/421_2.png) [@GIGObuster](https://boards.straightdope.com/u/GIGObuster)\ **Post date:** [November 20, 2006, 5:19am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/16 "2006-11-20T05:19:28Z")
[QUOTE=BrainGlutton] What makes you think the “gray goo” danger is overestimated? [/quote] I did check it, it is part of my book :). Suffice to say that the problem of getting the replication going to that level is to get a hold of enough **pure** or \*\*simple \*\* materials. AFAICR the reality is that you can get the nanites to a good source, but at that size even a good source will have many nano “monkey wrenches” or impurities to stop gray goo cold. > [@](#): > > If we can develop nanotech to the point where it has any beneficial uses, surely someone can develop it as a weapon . . . and one as difficult to control, once released, as biological weapons. As a weapon, yes, difficult to control, no. ---
**Author:** ![BrainGlutton](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@BrainGlutton](https://boards.straightdope.com/u/BrainGlutton)\ **Post date:** [November 20, 2006, 5:24am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/17 "2006-11-20T05:24:39Z")
[QUOTE=GIGObuster] I did check it, it is part of my book :). Suffice to say that the problem of getting the replication going to that level is to get a hold of enough **pure** or \*\*simple \*\* materials. AFAICR the reality is that you can get the nanites to a good source, but at that size even a good source will have many nano “monkey wrenches” or impurities to stop gray goo cold. As a weapon, yes, difficult to control, no. [/QUOTE] So, nanotech would not work at all except under controlled/prepared conditions? ---
**Author:** ![GIGObuster](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/gigobuster/32/421_2.png) [@GIGObuster](https://boards.straightdope.com/u/GIGObuster)\ **Post date:** [November 20, 2006, 5:42am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/18 "2006-11-20T05:42:05Z")
[QUOTE=BrainGlutton] So, nanotech would not work at all except under controlled/prepared conditions? [/QUOTE] If one excludes self replication, nanotech already does work outside the lab, One problem reported has been that some products are so stable that they are not biodegradable. So the issue of contamination has appeared, but many other produts are designed better and show lots of promise: [http://www.sfgate.com/cgi-bin/article.cgi?file=/c/a/2006/03/26/MNGP9HS9HJ1.DTL](http://www.sfgate.com/cgi-bin/article.cgi?file=/c/a/2006/03/26/MNGP9HS9HJ1.DTL) > [@](#): > > Nanotechnology could fix nerve, brain damage > Carey Goldberg, Boston Globe > > Sunday, March 26, 2006 > > In work that might hold promise for victims of spinal cord and brain injuries, researchers report that they have managed to restore sight to blinded hamsters using a process they call nanoknitting. > > The work represents the first time that nanotechnology – engineering on an ultra-tiny scale – has been used to fix brain damage, said Rutledge Ellis-Behnke, a neuroscientist at the Massachusetts Institute of Technology and the lead researcher on the paper. > > It will be years before the knitting technique can be tried in humans, and it must still prove its worth in other animal experiments, but Ellis-Behnke said he hopes it can someday serve as part of a multipronged therapy to help regenerate nerve connections in the millions of people who suffer strokes, spinal-cord damage and brain injuries, particularly those whose brain injuries leave them “no longer the person that was before.” > > “Our goal is to try to reconnect disconnected parts to restore that person as much as possible,” he said. > > The researchers injected nanofibers into the gap created by cutting the neural pathway that enables vision. These fibers linked themselves together into a tiny scaffold that apparently helped heal the brain tissue and let axons – the transmission lines that connect neurons – regrow. > > The knitting technique offers a possible way to overcome a formidable problem: When neural connections in the brain or spinal cord are damaged, they don’t tend to heal, which can result in lifelong brain damage and paralysis. When a neuron is cut, Ellis-Behnke said, it sprouts a “growth tip” like a tree whose branch has been cut, but then the growth stalls. Researchers are trying to figure out why and what they can do to spur more growth. ---
**Author:** ![msmith537](https://avatars.discourse-cdn.com/v4/letter/m/d9b06d/32.png) [@msmith537](https://boards.straightdope.com/u/msmith537)\ **Post date:** [November 20, 2006, 5:42am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/19 "2006-11-20T05:42:08Z")
“Hey baby…wanna kill all humans?” -Bender I imagine an AI would want whatever it was designed and built to want. A sentient robot car would want to get to it’s destinations. It would want to avoid damage to itself or injury to it’s occupants. It would “feel” uncomfortible when it developed mechanical trouble and would desire servicing. It may start to feel useless and depressed if it didn’t get driven enough. Now say such a car could override the users input if it was being driven in an unsafe and dangerous manner. Could it conceivably drive itself away if it felt it was being abused (say never washed or had its oil changed)? Could a sentient car develop a psychosis towards it’s owner and start acting more Christine than K.I.T.T.? I think that’s probably the more likely scenario than “KILL ALL HUMANS!” As our AI devices start becoming more self-aware, they may start behaving unpredictably as they try to fulfill their non-human needs. ---
**Author:** ![Mangetout](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mangetout/32/19_2.png) [@Mangetout](https://boards.straightdope.com/u/Mangetout)\ **Post date:** [November 20, 2006, 8:22am UTC](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081/20 "2006-11-20T08:22:15Z")
IMO, AI that merely ‘wants what we program it to want’ wouldn’t really be AI - it would just be an elaborate automaton, in fact I propose that the term ‘want’ is not really applicable to such a scenario. [Next page](https://boards.straightdope.com/t/what-would-a-sentient-robot-ai-really-want-out-of-life/381081.md?page=2)