What are the chances artificial intelligence will destroy humanity?
What, you thought the worst that could happen was a search engine declares its love for you? Then you haven’t been following the AI news very closely – specifically, the publication last year of a scholarly paper entitled “Advanced artificial agents intervene in the provision of reward.”
OK, that’s not as grabby as it might be. Lead author Michael Cohen punched it up some in a tweet: “Our conclusion is much stronger than that of any previous publication – an existential catastrophe is not just possible, but likely.” For those still not getting it, journalists covering the story dispelled all doubt: “Google and Oxford scientists claim AI might destroy mankind in future.” (Cohen is a doctoral candidate at the University of Oxford, a world center of what I’ll call scary-AI thought, and one of his co-authors is with DeepMind, Google’s AI research arm.)
This isn’t the first time claims of potentially homicidal machines have been made – they’ve been popping up for years. However, the impressive if sometimes creepy advances made in search engines lately have made the matter seem more urgent. I contacted Cohen to find out how dangerous AI really was.
I confess I wanted to discover the whole idea was batshit crazy. However, my exchange with Cohen made it clear this wasn’t a viable option. Here’s the best I could come up with: We can’t conclusively say artificial intelligent agents won’t exterminate us. But – and I realize some may not find this comforting – we can’t conclusively say they will.
How far off is Armageddon? Cohen and his co-authors are silent on this point. However, Nick Bostrom, founding director of the Future of Humanity Institute at Oxford and a leading light of the scary-AI school, compiled several surveys of predictions by AI researchers for his 2015 book Superintelligence. He found that the median prediction for when human-level machine intelligence would come to pass was 2040. Human-level machine intelligence is a gradation of artificial general intelligence or AGI, which is the gateway to artificial superintelligence (ASI) – and that’s when things start to look dicey for us in the meatspace.
How soon after AGI we get to ASI is unknown, but some argue it could be fast and unstoppable. If so, we needn’t worry about climate change, which Google in its AI-driven wisdom says will reach catastrophic levels by 2100. The killer robots will get us first.
This isn’t sounding promising, is it? So let’s see if we can whittle the problem down some, starting with predictions that AGI is likely to happen soon. What we’ve got now – sometimes referred to as artificial narrow intelligence (ANI) – mimics aspects of human intelligence, at times with disturbing fidelity. But no one seriously contends AI tools work the way the human mind does, because we don’t know how the mind works. Absent any real knowledge of what AGI will entail or what it’ll take to get there, predictions are just guesswork, regardless of who makes them.
One persuasive demonstration of this was a 2012 analysis of AI predictions by scientists Stuart Armstrong and Kaj Sotala. Their findings: (a) predictions varied widely, although AGI was most often forecast to arrive 15 to 25 years from whenever the prediction was made; and (b) the bell curve distribution for predictions by AI experts was almost identical to the one for non-experts. Conclusion: experts had as much insight into how soon AI would arrive as the average mope – i.e., none.
That’s about it for the good news. The bad news is that human annihilation is a plausible, if not certain, outcome of AI.
Here’s the reasoning, pieced together from my exchange with Michael Cohen, his paper, Nick Bostrom’s book Superintelligence, and the odd tidbit from the Internet, which I admit isn’t an approach I’d care to run past a Ph.D. board but may enable the lay reader to get the drift:
-
Suppose we create a machine with AGI. This machine doesn’t need to be brilliant; let’s say it has mere human-level intelligence. The main thing is, it’s capable of planning actions in pursuit of a long-term goal. We’ll call this machine the Agent.
-
Stop right there, you say. How do we know AGI is even possible? Harvard psychology professor Steven Pinker, a prominent AI skeptic, calls the concept of AGI “incoherent” and says all we can likely do is invent machines that mimic or surpass this or that human skill – in other words, bigger and better ANI devices.
-
To which I say: Sure, if Pinker is right, we can stop now and this will be a short column. But suppose he isn’t. One can make a plausible argument that we know AGI is possible because we’re examples of it – complex material systems that … well, “think” might be generous, but engage in whatever synaptic activity impels us to get off the couch and grab a beer. In principle there’s no reason this kind of thing couldn’t be replicated in silicon or other suitable medium, and technological progress and the profit motive being the inexorable forces they are, it seems certain we’ll get there someday. We just don’t know how soon.
-
Nonetheless, having read Michael Cohen, we know there’s a nonzero chance our Agent may interpret its instructions in twisted ways, go rogue, and snuff out humanity, when all we wanted it to do was sort machine screws in the Tesla plant. To head this off, we sharply limit the Agent’s contact with the outside world – for example, by only allowing it to print text to a screen.
-
Makes no difference. The Agent is so smart and devious it’ll be able to outfox its dimbulb human operators (us) and trick them into doing its bidding, such as hooking it up to the Internet.
-
That done, the endlessly resourceful and patient Agent will be able to recruit, purchase, or create unlimited numbers of assistants to implement its diabolical plans, up to and including the destruction of the human race.
-
You say: Wait a minute, I thought the Agent had AGI, specifically human-level intelligence. Now you’re telling me the Agent has superintelligence – ASI. How did that happen?
-
It’s a possible – some would say inevitable – consequence of AGI. If we read our Nick Bostrom, we learn that an Agent with AGI – and it doesn’t even need to have human-level intelligence; Bostrom says sub-human smarts would do – might nonetheless have a knack for coding and AI research, which would enable it to use its otherwise ordinary intellect to code a smarter version of itself, which in turn could code an even smarter version, and so recursively until voila, superintelligence and, some unknown time thereafter, supermalevolence.
-
So (you say) from a logical standpoint, by the mere act of conceding AGI is possible, we’ve signed up for the destruction of humanity?
-
That’s about the size of it.
-
That sucks.
-
I’m none too happy about it myself. However, on reflection, it seems to me the above-described train of reasoning contains some unacknowledged assumptions that, when subjected to close examination, could put a spanner in the works before humanity goes over the cliff.
-
Such as what?
-
We know general intelligence exists because we have it. We don’t know superintelligence exists.
-
Stephen Hawking was superintelligent.
-
Not to the level required by the doomsday scenario. The Agent is so super-duper brainy that it can fool everybody all the time, cook up plans that never fail, anticipate and compensate for everything that could go wrong, and outwit every attempt to defeat it. It can also summon endless resources (money, energy, materials), procure unlimited henchmen and other assistance, build any weapon in any quantity, and invent any desired technology (nanobots are a favorite among AI bloggers). And it would need all these abilities. You think extinguishing seven billion human lives would be easy?
-
So the Agent isn’t just really smart, in the sense of Einstein-type smart, it’s essentially a god. We get into what I call the “he’s Superman” argument, where the answer to every possible limitation on Superman’s powers is to say “he’s Superman.” In other words, we enter the realm of fantasy or, to put it in terms more suited to the academy, verge on an unfalsifiable claim.
-
That kind of talk will get you punched out at Oxford.
-
Right. More to the point, Michael Cohen was generous with his time, and it would be unseemly of me to repay the favor by dismissing the work of the scary-AI school. It’s not like anybody thinks artificial intelligence is a 100% benign technology. Sure, most of the problems evident so far are on the order of making it easier for college students to cheat. But while existential catastrophe may not be imminent, we can’t say it’s out of the question, and it’s certainly worth some advance thought. You never know. Maybe AGI won’t be achieved until a century from now, if ever. But it could be next Tuesday afternoon.
– CECIL ADAMS
After some time off to recharge, Cecil Adams is back! The Master can answer any question. Post questions or topics for investigation in the Cecil’s Columns forum on the Straight Dope Message Board, boards.straightdope.com/.
