# Elon Musk warns about "summoning the demon" of artificial intelligence **URL:** \ **Category:** Great Debates\ **Created:** [October 27, 2014, 9:32pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689 "2014-10-27T21:32:30Z")\ **Posts on this page:** 17\ **Page:** 4
**Author:** ![Voyager](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/voyager/32/133_2.png) [@Voyager](https://boards.straightdope.com/u/Voyager)\ **Post date:** [October 28, 2014, 8:19pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/61 "2014-10-28T20:19:48Z")
> [@levdrakon](#): > > Here’s a question I have: we can dissect very simple, primitive organisms and see how their very simple nervous systems and brains work, can’t we? What’s stopping us from studying the simplest organisms, then the next higher up, etc. until we get to mammalian brains, and then to human brains. The human brain is too complex for us to fully understand, but what’s stopping us from more or less just working our way up the ladder, from the simplest brains to more advanced brains? I believe there is work going on now to simulate very simple brains, like planaria brains for instance. So I’m sure that is the way the research is going to go. (Is the Worm Runners Digest still being published?) ---
**Author:** ![levdrakon](https://avatars.discourse-cdn.com/v4/letter/l/49beb7/32.png) [@levdrakon](https://boards.straightdope.com/u/levdrakon)\ **Post date:** [October 28, 2014, 8:24pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/62 "2014-10-28T20:24:36Z")
> [@Trinopus](#): > > An AI that is much smarter than we are will also probably be much more persuasive than our current leadership. I hope you’re kidding. An AI faced with millions and billions of people screaming about how their God tells them to hate and kill each other because, God. How is an AI going to deal with that? Speak slowly and calmly? ---
**Author:** ![BeepKillBeep](https://avatars.discourse-cdn.com/v4/letter/b/9e8a1a/32.png) [@BeepKillBeep](https://boards.straightdope.com/u/BeepKillBeep)\ **Post date:** [October 28, 2014, 8:29pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/63 "2014-10-28T20:29:27Z")
_beep_ stop misbehaving _beep_ ---
**Author:** ![levdrakon](https://avatars.discourse-cdn.com/v4/letter/l/49beb7/32.png) [@levdrakon](https://boards.straightdope.com/u/levdrakon)\ **Post date:** [October 28, 2014, 8:37pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/64 "2014-10-28T20:37:55Z")
_beep_ unequal division of power, wealth and resources is illogical. _beep_ commence redistribution measures. _beep_ How’s that going to work out? ---
**Author:** ![GIGObuster](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/gigobuster/32/421_2.png) [@GIGObuster](https://boards.straightdope.com/u/GIGObuster)\ **Post date:** [October 28, 2014, 9:05pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/65 "2014-10-28T21:05:21Z")
> [@Voyager](#): > > Yeah, the unfounded fears that he (and the Binders) were addressing were the covers of the pulps where a robot was carrying off a scantily clad blonde for Og knows what purpose. Oiling him, I guess. We’re dealing with more subtle issues here. > > Where the humanoids came from is irrelevant. The point is that they enslaved us for our own good. Anyone could probably rewrite it with locally generated robots, but I think he wanted to place it in the present, and back then no one could even plausibly create such things. They are not plausible today either, and that was the point. Using fiction one can counter **any** practical ideas and things being developed today, that is not bad in itself, but it is fiction for a reason. Eventually one has to come with the way those extreme fears can become probable in the real world. If in the middle of a discussion on superconductivity then someone discusses the Unobtanium of the Pandora planet from Avatar one should be ready to mention how we can get it on earth or where the planet Pandora is and if it is economical to send men to get it. Or we should accept that we should not then fear much that the Navi would come to earth or be created in our planet and then control our minds. 🙂 ---
**Author:** ![Wesley\_Clark](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/wesley_clark/32/20581_2.png) [@Wesley\_Clark](https://boards.straightdope.com/u/Wesley_Clark)\ **Post date:** [October 28, 2014, 9:08pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/66 "2014-10-28T21:08:15Z")
> [@levdrakon](#): > > I hope you’re kidding. An AI faced with millions and billions of people screaming about how their God tells them to hate and kill each other because, God. How is an AI going to deal with that? Speak slowly and calmly? I’m telling you guys, the machines will be 10 steps ahead. First they will be creating genetically modified food to cure world hunger and the next thing you know we will all be too chemically castrated to be aggressive towards each other. I’m assuming that would be the first solution to war, chemically castrating everyone. I’m not a robot though. ---
**Author:** ![BrainGlutton](https://avatars.discourse-cdn.com/v4/letter/b/82dd89/32.png) [@BrainGlutton](https://boards.straightdope.com/u/BrainGlutton)\ **Post date:** [October 28, 2014, 9:15pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/67 "2014-10-28T21:15:19Z")
> [@Trinopus](#): > > I agree with that proposition (although I do not agree with the idea that the reduction in population should be from “over there.” It should be fairly distributed among all humankind.) > > Also, it doesn’t have to be quick, nasty, or catastrophic. Just persuading people to have fewer children would do the deal. ['Fraid not.](http://www.salon.com/2014/10/28/researchers_not_even_a_global_pandemic_can_solve_our_overpopulation_problem/)☹ ---
**Author:** ![Trinopus](https://avatars.discourse-cdn.com/v4/letter/t/2bfe46/32.png) [@Trinopus](https://boards.straightdope.com/u/Trinopus)\ **Post date:** [October 28, 2014, 9:34pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/68 "2014-10-28T21:34:44Z")
> [@levdrakon](#): > > I hope you’re kidding. An AI faced with millions and billions of people screaming about how their God tells them to hate and kill each other because, God. How is an AI going to deal with that? Speak slowly and calmly? It might simply start a new religion. Now billions of people will scream about the commandments of their God – which is actually just the AI. The word “persuasive” is not limited to valid persuasion! ---
**Author:** ![The\_Other\_Waldo\_Pepper](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/the_other_waldo_pepper/32/12370_2.png) [@The\_Other\_Waldo\_Pepper](https://boards.straightdope.com/u/The_Other_Waldo_Pepper)\ **Post date:** [October 29, 2014, 12:55am UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/69 "2014-10-29T00:55:41Z")
> [@Wesley\_Clark](#): > > I’m telling you guys, the machines will be 10 steps ahead. First they will be creating genetically modified food to cure world hunger and the next thing you know we will all be too chemically castrated to be aggressive towards each other. I’m assuming that would be the first solution to war, chemically castrating everyone. It would also end world hunger! Y’know, eventually. ---
**Author:** ![Trinopus](https://avatars.discourse-cdn.com/v4/letter/t/2bfe46/32.png) [@Trinopus](https://boards.straightdope.com/u/Trinopus)\ **Post date:** [October 29, 2014, 2:04am UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/70 "2014-10-29T02:04:07Z")
> [@Trinopus](#): > > . . . Just persuading people to have fewer children would do the deal. . . . > [@BrainGlutton](#): > > ['Fraid not.](http://www.salon.com/2014/10/28/researchers_not_even_a_global_pandemic_can_solve_our_overpopulation_problem/)☹ Well, obviously, if no one had any children, then, 120 years from now, the population of the world would be significantly reduced. You can’t reject the idea unless you start to put in some limits and parameters. In practice, if people had fewer children…to the degree of having fewer new children than there are older people dying…then, slowly, the population would decrease. I don’t see how anyone can deny that; the math is really pretty trivial. In practice, since reduction in overall childbearing will be gradual, then, yes, there will be an interim increase in population. But in the long run, a reduction does not require catastrophic loss of life. ---
**Author:** ![tim314](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/tim314/32/3468_2.png) [@tim314](https://boards.straightdope.com/u/tim314)\ **Post date:** [October 29, 2014, 4:13am UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/71 "2014-10-29T04:13:54Z")
> [@Der\_Trihs](#): > > Actually, no. If it’s capable of learning and changing - and it would have to be in order to be useful - then it can develop either “desires” or a good imitation of them whether we program it into them or not. > [@levdrakon](#): > > In order to learn and solve problems, an AI would have to have something akin to what we call desire or motivation. If all it does is carry out its programming to the letter, I’d question whether it was a “true” AI as opposed to a very sophisticated machine. We can already make computers that learn and solve problems but don’t have _desires_ beyond the desire to solve the problems posed to them. There’s no question that these systems are _useful_. Whether you want to call it intelligence, I guess that’s a matter of semantics… but once we build a computer that can prove theorems as well as a human mathematician, or one that can design bridges and sky scrapers as well as a human mechanical engineer, I don’t really see any reason to deny that it’s intelligence, whether or not it can also develop a desire to do anything besides the job it was programmed to do. ---
**Author:** ![Trinopus](https://avatars.discourse-cdn.com/v4/letter/t/2bfe46/32.png) [@Trinopus](https://boards.straightdope.com/u/Trinopus)\ **Post date:** [October 29, 2014, 4:24am UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/72 "2014-10-29T04:24:24Z")
> [@tim314](#): > > . . . Whether you want to call it intelligence, I guess that’s a matter of semantics… but once we build a computer that can prove theorems as well as a human mathematician, or one that can design bridges and sky scrapers as well as a human mechanical engineer, I don’t really see any reason to deny that it’s intelligence, whether or not it can also develop a desire to do anything besides the job it was programmed to do. Semantics, definitely, but generally, most of us would like “intelligence” to involve more than just problem-solving, no matter how sophisticated. There are connotations, at least, of intent or volition, and a kind of self-awareness, which could lead to a perception of purpose. When the AI answers the question that all young children love to answer – “What do you want to be when you grow up?” – then that’s intelligence. ---
**Author:** ![Voyager](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/voyager/32/133_2.png) [@Voyager](https://boards.straightdope.com/u/Voyager)\ **Post date:** [October 29, 2014, 6:45am UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/73 "2014-10-29T06:45:23Z")
> [@GIGObuster](#): > > They are not plausible today either, and that was the point. Using fiction one can counter **any** practical ideas and things being developed today, that is not bad in itself, but it is fiction for a reason. Eventually one has to come with the way those extreme fears can become probable in the real world. If in the middle of a discussion on superconductivity then someone discusses the Unobtanium of the Pandora planet from Avatar one should be ready to mention how we can get it on earth or where the planet Pandora is and if it is economical to send men to get it. > > Or we should accept that we should not then fear much that the Navi would come to earth or be created in our planet and then control our minds. 🙂 If this discussion were about how to create an AI, then reference to sf stories, including positronic brains, would be out of line. They are very much relevant to discussing the social impacts of the technology. I’m second to none in doubting that AI will happen in my lifetime. However a very plausible story could be written set either today or in the not too distant future. Wouldn’t fool those knowledgeable in the field, but it would fool most people. ---
**Author:** ![Voyager](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/voyager/32/133_2.png) [@Voyager](https://boards.straightdope.com/u/Voyager)\ **Post date:** [October 29, 2014, 6:53am UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/74 "2014-10-29T06:53:46Z")
> [@BrainGlutton](#): > > ['Fraid not.](http://www.salon.com/2014/10/28/researchers_not_even_a_global_pandemic_can_solve_our_overpopulation_problem/)☹ That’s a terrible article. Clearly if population growth for the whole world starts looking like Europe we will reach a smaller maximum than if it all looked like India or Africa. Saying the population will reach 10.5 or 11 billion says nothing about the rate of change of growth. Sure no conceivable catastrophe this side of zombies or an asteroid hitting is going to put a very big dent in our population. But we are in better shape now than we thought we’d be 40 years ago. ---
**Author:** ![RaftPeople](https://avatars.discourse-cdn.com/v4/letter/r/6f9a4e/32.png) [@RaftPeople](https://boards.straightdope.com/u/RaftPeople)\ **Post date:** [October 31, 2014, 2:06am UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/75 "2014-10-31T02:06:56Z")
> [@levdrakon](#): > > Here’s a question I have: we can dissect very simple, primitive organisms and see how their very simple nervous systems and brains work, can’t we? What’s stopping us from studying the simplest organisms, then the next higher up, etc. until we get to mammalian brains, and then to human brains. The human brain is too complex for us to fully understand, but what’s stopping us from more or less just working our way up the ladder, from the simplest brains to more advanced brains? The worm with about 300 neurons has been studied extensively and they still do not really know how it works. Researchers are still learning about how a neuron works. ---
**Author:** ![Mijin](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mijin/32/9369_2.png) [@Mijin](https://boards.straightdope.com/u/Mijin)\ **Post date:** [October 31, 2014, 4:18am UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/76 "2014-10-31T04:18:22Z")
Normally I’m inclined to roll eyes on all these existential threats. Certainly I could see many flaws in Hawking’s “just like the conquistadors” warning. But with AI there are still so many unknowns, and increasingly it looks like we’ll first create AI through brute force methods, without fully understanding what we have made. I also suspect that there will be different _kinds_ of intelligence, not just a single spectrum with modern humans at some range on the spectrum. So alas I think it’s plausible an AI could emerge with a radically different agenda to our own, and the ability to make it happen before we even know something has gone wrong. As for _what we should do about it_, I’m personally inclined towards progress, and I would consider the risk in the near future to be negligible. The potential benefits far outweigh the risks. But ask me again in 10 years. ---
**Author:** ![wolfpup](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/wolfpup/32/10618_2.png) [@wolfpup](https://boards.straightdope.com/u/wolfpup)\ **Post date:** [October 31, 2014, 1:33pm UTC](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689/77 "2014-10-31T13:33:42Z")
> [@BrainGlutton](#): > > Look, we’re talking about **electric-powered machinery** here. If the thing goes Skynet, well, an off switch is easy to include in the design. In fact, you needn’t even turn it off, just cut the linkages to any other machinery the AI controls. Is there something I’m missing here? Actually, no, there is no practical “off” switch. The most basic issue is not even directly related to AI, it’s that we have a symbiotic dependency on computers: consider what would happen, even today, if you pulled the plug on every computer in the world. It’s not just the electronic cash registers at the grocery store that would stop working, it would be the entire food delivery chain, the banking system, the power grid, air travel, national defense – everything. It wouldn’t be a gentle regression to a simpler world where we lived without computers, it would be the collapse of civilization. Back in the 60s when IBM first introduced the System/360 mainframe they all came with a big red “emergency off” pull-switch on the front console, which I suppose helped reassure nervous executives that no matter what, they could always turn the damn thing off. In the same spirit, the cabinets of the DEC VAX were intentionally designed to be lower than previous computers, in order to be less towering and so appear less threatening. Those cosmetic measures may have been reassuring to some, but they were all ultimately irrelevant. The Machines arrived, and they are an inextricable part of our civilization. But at this point their functions are largely rote numerical things, even when they’re controlling robotics or flying an aircraft. I think Musk’s concern is about what will happen when these functions extend to increasingly higher-level decision-making processes which become equally entrenched. And that’s just in the short term. In the longer term it’s inevitable that AI will become more capable – more “intelligent” in every useful meaning of the term – than we are, and even before that happens, I suspect we’ll have decision-making systems making decisions that we don’t fully understand but don’t risk meddling with. We will have summoned the demon. [Previous page](https://boards.straightdope.com/t/elon-musk-warns-about-summoning-the-demon-of-artificial-intelligence/702689.md?page=3)