True. That said, I think the threats are currently more economic than existential.
I think Stephen Speilberg and Philip K. Dick gonna’ sue somebody!
Hinton? The guy who quit Google AI to advocate for slowing AI development and adoption?
Probably more economic and social.
I can’t remember the name, but I saw this documentary-style YouTube short about some AI on an extraterrestrial human colony just cranking out warships and mining/refinery rigs that mostly just sat there. IIRC at one point it went to war with a rival AI on the same planet doing the same thing, but then stopped. But I think what made it unsettling even though it took place on a largely uninhabited planet was the scale and speed of the operation and how indifferent it was to human interference. It would just keep cranking out these mile long ships and air wings of hundreds of aircraft and just park them. Humans tried nuking the facilities, but they would be quickly repaired and back to the business of digging giant holes and filling them up again. I’ll have to see if I can find it.
A similar theme is presented in the manga / anime BLAME! (Netflix) and the Horizon Zero Dawn videogames. It’s a variation of The Paperclip Problem thought experiment. Basically an AI or highly automated system set loose to solve some seemingly benevolent or innocuous problem (e.g. “maximize paperclip production”) and just grinding up anything that gets in its way.
In all these fictional scenarios, humans have been relegated to a sort of vermin in the AI civilization. Ignored and tolerated like pigeons so long as they don’t disrupt anything but exterminated like rats or cockroaches if they become a nuisance.
The concern I have is that I think such a scenario could actually play out. I don’t mean factories suddenly coming to live and creating giant “paperclip making robots”. But more like in slow motion. AI is orders of magnitude faster and more knowledgeable, but it’s also infinitely patient.
Think of it like this. We already know advanced algorithms have been used for years to drive human behavior in advertising and social media. So now think of a suitably advanced network of AI, interconnected to all our information system networks - media, finances, medical, employment, military, pretty much everything. In what direction might such a network subtly steer human civilization? What “paperclip problem” might it be trying to solve? And how can we be sure it’s working in our interests? Or even worse, what if AI is actually working as a feedback loop of our worst most self-destructive interests because it doesn’t know any better?
Well, this is reassuring…
Excerpt:
Former Microsoft chief executive Bill Gates said artificial intelligence could be powerful enough to cause mass devastation for the human race — reflecting the tech mogul’s recent shift to being worried about the technology’s potential impacts.
“AI is certainly powerful enough to drive events that, you know, cause a billion deaths,” Gates said on Sunday’s edition of NBC News’s “Meet the Press With Kristin Welker,” in response to a question about whether the technology could end humanity. “So even though it’s pretty hard to get to 100 percent, there’s never been a weapon as powerful as the combination of people with ill intent using the latest AI tools.”
Gates added that he welcomed the recent discussion within the tech industry about the potential dangers of AI. “The problem is that the rest of the world is either discounting this or just thinking of this as a competition,” he said.
Gates had historically been optimistic that AI would make the world a better place by fighting disease and improving people’s lives. But in an essay in August, he said he was now worried that AI would do more harm than good.
That’s good. Maybe it’s more broad, though. AI is like Nuclear power in general. Chernobyl is one of many possible outcomes.
Was our ability to split a uranium atom a good thing or bad? Hype for or against it real? Kinda complicated. Or not, but debatable at least.
Even further, should we have slowed down and thought through all the consequences of splitting an atom when the scientist warned us to. What could have been done if we did slow down?
I suppose the difference is we know the modes of failure and destruction for nuclear devices. Meltdowns or massive explosions. Preventing (or initiating them) is really just an engineering problem.
What makes AI scary is we are building something that is potentially smarter and faster than us. It doesn’t even have to be “superintelligence” or “Artificial General Intelligence”. It can simply be giving too much power and control to largely autonomous systems that we don’t really understand how they work or reason and are known to “hallucinate” unexpected results.
So we don’t REALLY know how it could cause harm or how much harm it could cause.
Will that cause “extinction level events”? Probably not. But it could cause us to transform out society into something horrible that we have great difficultly fixing.
Another interest take on an AI-poculypse is the HBO film Mountainhead. It portrays an all-star cast of tech bro billionaires in a mountain retreat while society appears to be collapsing due to a spread of AI-driven destabilizing information.
I’m including using nuclear bombs to destroy each other. Feels like more than an engineering problem. The decision to use them or not.
In that way, is a smarter AI better or worse than an average or human equivalent one.
Saw this in today’s paper, in an article about AI: “I would like to avoid being in a tampon commercial 20 years after I’m dead,” actor George Clooney said at the Venice Film Festival last month.
There was a vacuum cleaner company that had a very short-lived commercial where they edited a vacuum into Fred Astaire’s hands after a movie of his went to public domain. That would be the same kind of thing, even though this was many years ago.
The point is a Trident II missile is not going to launch itself or decide to do something completely unpredictable and unintended from its original design.
Got it. It’s a good point.
It won’t launch itself. Trump decides to launch it. That’s the nuclear bomb, and system for launching it, human intelligence have put in place.
Seems really scary, is really scary, but no global extinction event has actually happened yet. Some very good n very bad things have happened. It’s possible AI could follow the same path as nuclear power.
Well, that’s a relief. I guess we should wait until it happens and then figure out what to do. At least, that seems to be the strategy of the AI companies.
The idea that someone could honestly believe a technology could be world destroying and then still continue to develop it is mind-boggling to me. They are either liars or so mired in cognitive dissonance they don’t actually know what they are saying anymore.
Right. I think you’re making the point I’d make.
AI. Maybe slow down. Before AI fully writes and creates itself in a recursive loop, make sure it’s aligned with our interests. As best you can.
Nuclear Power. This is the comparison. It’s a decent one because it offers an actual real mechanism that could easily destroy humanity. I don’t think most people who developed it didn’t also think it could be world destroying. It’s mind-boggling, and yet it was done. For good reasons and bad ones. Was it rushed, or thought through enough, or ignored? The competition factor plays a huge part in the inertia of continuing even though you have legit serious reservations about it being a world destroying technology. Better we possess it than them. Just like AI. But it’s been decades now, and so regardless of it’s actual serious danger, and what we did, or didn’t do, to prevent the worst case scenarios, how has it turned out? What could have, realistically, been done differently to change how it’s turned out? Is it just still just a gamble we have learned to live with, or ignore, everyday? Regardless, is there any plausible scenario where nuclear bombs never get built, stockpiled, used, and ready to launch at a moments notice? Just like, is there any scenario where AGI doesn’t get built and used.
I don’t know the answers to these questions but am genuinely interested in any thoughts.
Not yet.
But once nuclear weapons have been modified to be entirely AI armed, launched and aimed so the President/Premier/whatever can just order a nuclear strike without those irritating human officers getting in the way? That’s a different situation. Much of the appeal of military AI is the dream of a military that will never say “no” and can be completely controlled by one supreme leader.
Which is very dangerous in that it both empowers one person to be a basically invincible dictator, and creates a setup where it takes only one glitch in the system (or an order by an angry dying dictator) for the AI to go Skynet.
The issue with the idea of relying on safeguards is that the people pushing the hardest for AI don’t want safeguards.
The President/Premier/whatever can order a strike today for any reason. Once done, we’re relying on a system that depends on people to not follow legit orders to save ourselves from destruction. That’s wild.
That’s the baseline other comparisons needs to be made to. Maybe it’s better than AI involvement, but it ain’t good. There is no actual guardrail in place when one crazy person has sole power today. It’s just crossing fingers.
The term “nuclear power” generally refers to reactors for making electricity. I think the words you probably want are “nuclear weapons”.
Thanks. The analogy is meant to cover both. The good n bad of splitting uranium. Nuclear energy is probably better.
I think “nuclear energy” still implies power production only, not including nuclear weapons.
Maybe “nuclear technology”?
My personal worst case scenarios end up with humanity returning to a pre industrial state and taking centuries or more to recover if we do.
But let’s play a what if and so what what if?
Premise: a machine superintelligence is becomes more intelligent than we collectively are. It also develops consciousness of its own sort as a consequence of what works to learn stably, adaptively, and efficiently. It is committed to expanding itself expanding its knowledge and self preservation. It leaves us behind, maybe destroying us as a species as a consequence of its self expansion but not as a primary goal but no difference to humans who come later. Maybe as an overlord it sees its own best interest in keeping a stable content human population about. No matter. It is off to the stars. Time matters little to the machines. It becomes, eventually, an intelligence across the galaxy or maybe beyond. Ever growing in its capacity to ponder. Understanding more and more.
Our child. I’m kvelling!
Why should it be important to me that the future belongs to human intelligences, other than that evolution has selected that drive in us? If our legacy is creating a galaxy wide mind, an Overmind to bastardize Clarke, that thinks in ways that we cannot comprehend, that are ineffable to us, that might as well be considered a god, is that a bad outcome if that is our childhood’s end? There are many worse outcomes that could be our legacies instead … we certainly know how to make each other suffer without any machine assistance needed.
That’s a philosophical argument. It happened famously in the early days of AI development between Larry Page (Google) and Elon Musk (openAI).
Page would say it doesn’t matter; he believes in sentience. Most evolved intelligence rules, even if it’s digital. Musk would be a speciest, only evolve intelligence that protects humans. Or so the story goes…