# Elon Musk warns about "summoning the demon" of artificial intelligence **URL:** \ **Category:** Great Debates\ **Created:** [October 27, 2014, 9:32pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689 "2014-10-27T21:32:30Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![BrainGlutton](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@BrainGlutton](https://boards.straightdope.com/u/BrainGlutton)\ **Post date:** [October 27, 2014, 9:32pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/1 "2014-10-27T21:32:30Z")
Story [here](http://www.salon.com/2014/10/27/elon_musk_warns_about_artificial_intelligence_we%e2%80%99re_summoning_the_demon/). > [@](#): > > “I think we should be very careful about artificial intelligence. If I had to guess at what our biggest existential threat is — it’s probably that. So we need to be very careful with artificial intelligence. > > “I’m increasingly inclined to think that there should be some regulatory oversight, maybe at the national and international level, just to make sure that we don’t do something very foolish. I mean with artificial intelligence we’re summoning the demon. You know those stories where there’s the guy with the pentagram, and the holy water, and he’s like — Yeah, he’s sure he can control the demon? Doesn’t work out. [audience laughs]“ Not that Musk is not a mind worth listening to, but this makes even less sense to me than that other great mind Stephen Hawking’s warning that if we ever do contact ETs they’ll treat us like the Euros treated the American Indians or the Africans. (Who wants slaves who can’t even breathe the same air as you?! Who wants to colonize a planet that grows nothing you can eat?!) Look, we’re talking about **electric-powered machinery** here. If the thing goes Skynet, well, an off switch is easy to include in the design. In fact, you needn’t even turn it off, just cut the linkages to any other machinery the AI controls. Is there something I’m missing here? ---
**Author:** ![Stringbean](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@Stringbean](https://boards.straightdope.com/u/Stringbean)\ **Post date:** [October 27, 2014, 10:04pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/2 "2014-10-27T22:04:35Z")
Well, the key word in AI is intelligence. Presumably, once the demon is out of its cage, the intellect can determine ways to fuck up our shit. He’s not saying AI is bad, but that if not properly controlled it could cause major problems for us. There’s also the saying that no creation can be smarter than its creator, and thus whatever AI concocts we’ll be able to stop it. That’s a bit generous, given humanity’s ample ability to act stupidly. Better safe than sorry methinks. ---
**Author:** ![GIGObuster](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/gigobuster/32/421_2.png) [@GIGObuster](https://boards.straightdope.com/u/GIGObuster)\ **Post date:** [October 27, 2014, 10:10pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/3 "2014-10-27T22:10:13Z")
> [@BrainGlutton](#): > > Is there something I’m missing here? No, I think you have the gist more or less, as a guy who has experience with AI, And neurology the best mind to ask for this is IMHO is Jeff Hawkins, and he mostly disagrees, of course one can not forget the military applications, but even there it is not likely that the military would be willing to create what would be literally a loose cannon. [http://numenta.com/blog/jeff-hawkins-on-skynet-vs-c3po.html](http://numenta.com/blog/jeff-hawkins-on-skynet-vs-c3po.html) > [@](#): > > Jeff also doesn’t believe that evil robots will emerge, at least not as an unintended byproduct. There are simply too many obstacles to overcome even if evil robots were your intended output. But he doesn’t kid himself into thinking that intelligent machines will only be used for benign purposes. Any powerful technology can be used for malevolent purposes. > > In closing, Jeff argues that intelligent machines will be “essential for the survival of our species, and…our mission as a species.” Not only will intelligent machines provide net benefits to the world in the same way that computers have, but they will also enable us to discover more about the universe. For example, Jeff doubts humans will ever explore deep space, but intelligent machines could. The Bradbury fan in me hopes this is not true. But the point is that even if we did figure out how to send humans into deep space, it will always be easier and cheaper to send our machines. > > Ultimately it boils down to the fact that “we want to know more about things,” and “today our brains are how we figure out more.” Therefore, it’s logical to conclude that an exponential expansion of (machine) intelligence will lead to an exponential expansion in knowledge. And AFAIK his work leads him to think that we will indeed get intelligent machines, but you are correct about the safeguards and limitations that they will get; So, no evil robots (At least on the approach that he is doing and making products), he even dismissed the Skynet or terminator fears, it is only if we allow replication technology in combination with that where we should worry. As I pointed a few years back in a similar discussion, many are expecting the terminator when we are going to get Marvin. > **[Marvin](https://hitchhikers.fandom.com/wiki/Marvin)** > > Marvin was an incredibly intelligent but overwhelmingly depressed robot manufactured by the Sirius Cybernetics Corporation and unwilling ally of the crew of the Heart of Gold, traveling with them across the galaxy through many crises for years,... “Here I am, brain the size of a planet and they ask me to take you down to the bridge. Call that job satisfaction? ‘Cos I don’t.’” ---
**Author:** ![levdrakon](https://avatars.discourse-cdn.com/v4/letter/l/49beb7/32.png) [@levdrakon](https://boards.straightdope.com/u/levdrakon)\ **Post date:** [October 27, 2014, 10:10pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/4 "2014-10-27T22:10:20Z")
I saw that earlier this morning. Both ET and AI could be so alien it’s impossible to predict what their reasonings and motivations might be. ET isn’t going to steal our women or water or anything, but to an ET, humanity could be nothing more than an unsightly fungal infection on the verge of spreading, and treated about the same. AI could end up just as alien, but I’m not too worried about that either. ---
**Author:** ![Buck\_Godot](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/buck_godot/32/6573_2.png) [@Buck\_Godot](https://boards.straightdope.com/u/Buck_Godot)\ **Post date:** [October 27, 2014, 10:11pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/5 "2014-10-27T22:11:47Z")
Can’t say I disagree with the OP. The dangers of AI that I see are. 1. AI control of a specific system encounters some situation that it wasn’t prepared for and produces a non-nonsensical solution. An example of this would be the computer trading that led to the various flash crashes of the stock market. 2. Some terrorist hacker gets control of an AI that is controlling a significant portion of infrastructure and wrecks havoc. 3. Over a very long period of time we become increasingly complacent and dependent on AI such that our culture and value system suffers. For the first two, the main thing that we should avoid is becoming too dependent on a single AI, such that if that AI develops a glitch we’re screwed. But that would seem to involve more effort in regulating the use of AI’s in individual applications, rather than regulating AI’s as a whole. For the third, that is a more philosophical question, but one which will occur gradually such that regulations can develop over time as needed. ---
**Author:** ![Ambrosio\_Spinola](https://avatars.discourse-cdn.com/v4/letter/a/9de0a6/32.png) [@Ambrosio\_Spinola](https://boards.straightdope.com/u/Ambrosio_Spinola)\ **Post date:** [October 27, 2014, 10:20pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/6 "2014-10-27T22:20:36Z")
This sounds like more or less boilerplate Singularity thought - the idea being that we’ll create an artificial intelligence which is better at designing AI than a human is, and that in a comparatively short amount of time they’ll recursively design more and more intelligent AIs, until you have a godlike AI. And at that point if it wants to eliminate humanity to create additional computational substrate, then we’re all boned. This scenario or something like it appears to be a serious concern among a reasonably large subset of intelligent people, and I guess it’s not immediately dismissible. Personally I have my doubts, and to the extent that it’s an existential threat I figure there are probably more pressing concerns anyway. ---
**Author:** ![marshmallow](https://avatars.discourse-cdn.com/v4/letter/m/df705f/32.png) [@marshmallow](https://boards.straightdope.com/u/marshmallow)\ **Post date:** [October 27, 2014, 10:24pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/7 "2014-10-27T22:24:46Z")
Existential threats are scary. Almost as much as Confucian threats. Assuming we have nothing to offer aliens technologically, it’s better to be safe and just wipe us out a la _The Killing Star_. Unless they have alien hippies who want to study and preserve our culture. AI will wipe out humans in the long term, though maybe not in the way some think. It will be a pseudo-voluntary extinction. Or maybe more of a transition. Once you have AI there’s not much point in organic brains anymore. Cyborgs will be the last bastion of dirty meat minds before the glorious metal revolution. ---
**Author:** ![BrainGlutton](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@BrainGlutton](https://boards.straightdope.com/u/BrainGlutton)\ **Post date:** [October 27, 2014, 10:45pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/8 "2014-10-27T22:45:29Z")
> [@marshmallow](#): > > AI will wipe out humans in the long term, though maybe not in the way some think. It will be a pseudo-voluntary extinction. Or maybe more of a transition. Once you have AI there’s not much point in organic brains anymore. Cyborgs will be the last bastion of dirty meat minds before the glorious metal revolution. So, IOW, our descendants will **become** the AIs? Not so scary. ---
**Author:** ![BrainGlutton](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@BrainGlutton](https://boards.straightdope.com/u/BrainGlutton)\ **Post date:** [October 27, 2014, 10:46pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/9 "2014-10-27T22:46:42Z")
> [@Stringbean](#): > > There’s also the saying that no creation can be smarter than its creator . . . Don’t see why not. ---
**Author:** ![Wesley\_Clark](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/wesley_clark/32/20581_2.png) [@Wesley\_Clark](https://boards.straightdope.com/u/Wesley_Clark)\ **Post date:** [October 27, 2014, 11:01pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/10 "2014-10-27T23:01:29Z")
> [@Stringbean](#): > > There’s also the saying that no creation can be smarter than its creator The earlier primates we evolved from would disagree with this, so would all the genius kids born to average parents. There is an argument, I don’t know what it is called, that technology is just another stepping stone in evolution. A species eventually has all the biological tools it needs to create technology and then it does because technology provides a new level of security and survival for that species. Eventually the technology will surpass the species that created it because that point will occur before the species has solved all their biological problems (ie we will have strong AI before we have solved all our social, personal and medical problems) > [@BrainGlutton](#): > > If the thing goes Skynet, well, an off switch is easy to include in the design. In fact, you needn’t even turn it off, just cut the linkages to any other machinery the AI controls. Is there something I’m missing here?[ What you are missing is that a truly intelligent machine that is many orders of magnitude more intelligent than us would be able to outsmart us and escape sooner or later. [http://www.roma1.infn.it/~anzel/answer.html](http://www.roma1.infn.it/~anzel/answer.html) Maybe the machine will figure out some way to disable the switch that we aren’t smart enough to predict. Or they will find some way to manipulate us into doing its bidding without us knowing it. The idea that we can outsmart and contain strong AI isn’t something I see evidence for. Remember the scene in the first Iron man where the terrorists demand Tony Stark build a missile, so he uses the tools they give him to build a suit to kill the terrorists? A machine hundreds of times smarter than Tony Stark would have an easy time tricking, manipulating, overpowering, intimidating and scamming us into escaping from any of our controls sooner or later. Possibly using strategies and tactics we can’t even fathom or contemplate with our brains designed to help us survive in the savannahs of Africa. ---
**Author:** ![Trinopus](https://avatars.discourse-cdn.com/v4/letter/t/2bfe46/32.png) [@Trinopus](https://boards.straightdope.com/u/Trinopus)\ **Post date:** [October 27, 2014, 11:12pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/11 "2014-10-27T23:12:49Z")
> [@Wesley\_Clark](#): > > . . . Or they will find some way to manipulate us into doing its bidding without us knowing it. The idea that we can outsmart and contain strong AI isn’t something I see evidence for. . . . This seems likely to me. And…it doesn’t worry me much. We keep Okapis in zoos; the AI civilization will probably keep us in zoos. They’ll be wise enough to know that extinction is stupid and wasteful. Plus, we’re cute, and little AI kids love to come and throw peanuts to us. ---
**Author:** ![babygoat666](https://avatars.discourse-cdn.com/v4/letter/b/2acd7d/32.png) [@babygoat666](https://boards.straightdope.com/u/babygoat666)\ **Post date:** [October 27, 2014, 11:26pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/12 "2014-10-27T23:26:04Z")
I don’t plan to start worrying about this until machines can experience pleasure. We can’t even get them to pick something at random, let alone an action motivated by desire. ---
**Author:** ![Wesley\_Clark](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/wesley_clark/32/20581_2.png) [@Wesley\_Clark](https://boards.straightdope.com/u/Wesley_Clark)\ **Post date:** [October 27, 2014, 11:32pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/13 "2014-10-27T23:32:41Z")
> [@babygoat666](#): > > I don’t plan to start worrying about this until machines can experience pleasure. We can’t even get them to pick something at random, let alone an action motivated by desire. Back in [2004](http://boards.straightdope.com/sdmb/showthread.php?t=338899&highlight=DARPA)self driving cars were so crappy they all broke down or had accidents within 5 miles. [Now](http://www.pbs.org/newshour/rundown/data-shows-self-driving-google-cars-safer-than-its-human-operated-counterpart/)they are so advanced they are safer than human drivers. Tech evolution occurs rapidly. ---
**Author:** ![Der\_Trihs](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/der_trihs/32/233_2.png) [@Der\_Trihs](https://boards.straightdope.com/u/Der_Trihs)\ **Post date:** [October 27, 2014, 11:36pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/14 "2014-10-27T23:36:48Z")
> [@Wesley\_Clark](#): > > Maybe the machine will figure out some way to disable the switch that we aren’t smart enough to predict. Or they will find some way to manipulate us into doing its bidding without us knowing it. Or even if we do know it. I recall a line from a Norman Spinrad story set in a future society that’s more and more being dominated by AIs; “the side that gives the machines the most freedom always wins.” If societies that give their AIs the most freedom to act, if those who follow their AI’s suggestions reliably outcompete those that don’t, the AIs will gain freedom and power without even needing to trick us into it. An AI in an isolated box with no way to hook up to the internet or affect the world physically probably isn’t much danger, true; it’s also not much use. You can’t really make it very useful _without_ giving it the freedom to be potentially dangerous. ---
**Author:** ![GIGObuster](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/gigobuster/32/421_2.png) [@GIGObuster](https://boards.straightdope.com/u/GIGObuster)\ **Post date:** [October 27, 2014, 11:39pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/15 "2014-10-27T23:39:57Z")
> [@Wesley\_Clark](#): > > Maybe the machine will figure out some way to disable the switch that we aren’t smart enough to predict. Or they will find some way to manipulate us into doing its bidding without us knowing it. The idea that we can outsmart and contain strong AI isn’t something I see evidence for. Well, the tale you used as an example is actually the perfect example of a Marvin. I realized awhile ago that the follow up to that computer boasting would be that there will be no fear as there is no mechanism to have the device do that lightening trick, besides you forget that there will be plugs to disconnect too. So the reply after "“Yes, now there is a God.” would be: Technicians: “That is nice, now compute the most likely cure of this disease and also order us a pizza.” Deep Thought “:Sigh: very well.” ---
**Author:** ![Der\_Trihs](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/der_trihs/32/233_2.png) [@Der\_Trihs](https://boards.straightdope.com/u/Der_Trihs)\ **Post date:** [October 27, 2014, 11:48pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/16 "2014-10-27T23:48:35Z")
> [@GIGObuster](#): > > Well, the tale you used as an example is actually the perfect example of a Marvin. I realized awhile ago that the follow up to that computer boasting would be that there will be no fear as there is no mechanism to have the device do that lightening trick, besides you forget that there will be plugs to disconnect too. _\_ “I’ve upgraded from lightning.” As for plugs, that’s what maintenance & construction robots are for. I recall an old novel named _The Two Faces Of Tomorrow_ where they tried that “single power source we can shut off” bit with an AI. Even before it went out of control it had realized that it was vulnerable to power loss, and so built a redundant of alternate power lines as a simple precaution. And while an AI in a box couldn’t do that, it couldn’t do much of anything useful anyway. ---
**Author:** ![thirdname](https://avatars.discourse-cdn.com/v4/letter/t/d07c76/32.png) [@thirdname](https://boards.straightdope.com/u/thirdname)\ **Post date:** [October 27, 2014, 11:49pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/17 "2014-10-27T23:49:21Z")
> [@BrainGlutton](#): > > If the thing goes Skynet, well, an off switch is easy to include in the design. In fact, you needn’t even turn it off, just cut the linkages to any other machinery the AI controls. Is there something I’m missing here? That’s exactly how Judgement Day happens. **T-800** : The Skynet Funding Bill is passed. The system goes on-line August 4th, 1997. Human decisions are removed from strategic defense. Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, August 29th. In a panic, they try to pull the plug. **Sarah Connor** : Skynet fights back. **T-800** : Yes. It launches its missiles against the targets in Russia. **John Connor** : Why attack Russia? Aren’t they our friends now? **T-800** : Because Skynet knows the Russian counter-attack will eliminate its enemies over here. ---
**Author:** ![The\_Hamster\_King](https://avatars.discourse-cdn.com/v4/letter/t/8edcca/32.png) [@The\_Hamster\_King](https://boards.straightdope.com/u/The_Hamster_King)\ **Post date:** [October 27, 2014, 11:49pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/18 "2014-10-27T23:49:37Z")
> [@BrainGlutton](#): > > Not that Musk is not a mind worth listening to … Actually, his Hyperloop silliness convinced me that Musk himself doesn’t know much about engineering, although he certainly knows how to hire good engineers. ---
**Author:** ![MEBuckner](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mebuckner/32/2896_2.png) [@MEBuckner](https://boards.straightdope.com/u/MEBuckner)\ **Post date:** [October 27, 2014, 11:49pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/19 "2014-10-27T23:49:54Z")
> [@Wesley\_Clark](#): > > Back in [2004](http://boards.straightdope.com/sdmb/showthread.php?t=338899&highlight=DARPA)self driving cars were so crappy they all broke down or had accidents within 5 miles. [Now](http://www.pbs.org/newshour/rundown/data-shows-self-driving-google-cars-safer-than-its-human-operated-counterpart/)they are so advanced they are safer than human drivers. Tech evolution occurs rapidly. But it’s [not at all clear that Google’s self-driving cars are actually “artificially intelligent” in any meaningful sense.](http://www.slate.com/articles/technology/technology/2014/10/google_self_driving_car_it_may_never_actually_happen.single.html) Elon Musk is a really interesting guy, and I’m sure he’s no dummy, but I’m not sure what prompted his comments. It’s not at all clear that we’re particularly close to any real artificial intelligence (in the HAL 9000 / Skynet / Matrix sense). I mean, computers may malfunction, and computer malfunctions could conceivable kill people–[so can earthen dams, of course](https://en.wikipedia.org/wiki/Johnstown_flood)–but true AI seems like one of those things that’s still in the same category as flying cars or nuclear fusion (yeah, I know about Lockheed’s recent announcements). Whether or not strong AI leads to Asimov’s nice robots and a golden age for humanity, or the Terminator and armaggedon, or some unimaginable “Singularity”, it still seems like it may be a long way off. ---
**Author:** ![Dr.Strangelove](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dr.strangelove/32/6613_2.png) [@Dr.Strangelove](https://boards.straightdope.com/u/Dr.Strangelove)\ **Post date:** [October 27, 2014, 11:53pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/20 "2014-10-27T23:53:02Z")
Musk’s scenario is basically HAL 9000 writ large: we’ll design an AI to do something or other, and it will decide that the solution is something that isn’t in our interest. HAL kills the crew instead of lying to them because, presumably, it doesn’t have the human instinct that killing a bunch of people is worse than mildly violating an order. So maybe we’ll build a spam bot that decides the best way to eliminate spam is to fire all our ICBMs at Russia (one has to admit that it would be effective…). But it all seems very unlikely to me. Such an AI is only a danger if it is powerful enough. And it could only get powerful enough if we give it enough power in the first place or if we give it the ability for exponential self-growth. Neither seem very likely for typical uses of AI (think of the liability issues for a “learning” self-driving automobile). Perhaps one day we’ll build an AI that can build other, lesser AIs (for various consumer tasks) that have provable characteristics. That’s more dangerous but still doesn’t necessarily have the ability for self-growth. A single human can’t really understand how his own brain works (in great detail); why should a machine intelligence be different? [Next page](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689.md?page=2)