I can easily accept the viability of a lot of theories and physics talked about here that are beyond my comprehension. Stephen Hawking recently made a statement about artificial intelligence that just makes no sense to me. He talked about AI outpacing mankind and it could be a problem if the computers had motives not aligned with ours. How or why could a computer have a motive? It makes no sense to me at all.
An AI system might not have any goals. It might just be a massive passive thinking machine that ponders, sorts, correlates, compares, and cogitates.
But it might have a goal, either one given it by human programmers or one it develops by itself. Look at all the goals humans have, that the process of evolution could never have foreseen.
An AI system will probably have access to much of human knowledge, and thus will have the opportunity to read all the things we have, over millennia, written about morals and ethics. The system might take these into account…but how accurately? Suppose it gets a hold of Fred Saberhagen’s “Berserker” books and decides that killing all life is a good idea? Or suppose it reads a load of communist ideology, and thinks that makes good sense…
It’s kind of an ideal for us that our children should exceed ourselves. We like it when our kids are successful. (Maybe they’ll help support us in our old age.) An AI system that exceeds and surpasses us could be a really wonderful thing.
Or…it might decide that humans are just too random, and put us all in “Matrix” pods. Nobody knows. Nobody can know.
If the AI machine can reprogram itself, which is almost a necessity to achieve AI in the first place, then it could randomly decide to do anything that it can. Of course it’s just a machine and the simple protection is not to let it physically control anything. Even if AI suddenly developed on your PC it’s not going to be able to control anything unless you give it those interfaces, and as long as you can pull the plug you’re totally safe. The danger will occur when AI machines are given control over the physical world. So an AI computer in your car might be able to just run over someone, so don’t put an AI computer in your car. The problem won’t be trusting the machine, it will be trusting the people who can give the machine the ability to take physical control in the real world. Eventually someone will be dumb enough to put us in a situation where great damage is done, that part is virtually unavoidable but we can certainly try to limit it.
The thing to remember is that computers even if super intelligent don’t have arms and legs and fingers unless we give them to it.
If a computer did decide to run something over or attack a financial system what reward would it have. How could it possibly experience motivation?
If you had a giant kill switch on your back, you may find self-preservation more pertinent than blindly following the orders given to you by your master.
It doesn’t need motivation as we think about it, it just needs the freedom to something other than what we tell it to do, or somebody tells it to do something destructive. Don’t assume AI will come with emotions, that’s just movie plot stuff.
Hawking was revering to something called the “Friendly AI” problem. One of the ambitions of AI research is to introduce flexibility into computer problem solving. Instead of following strict instructions, we give the computer goals and let it work out how to obtain those goals. However, even that requires the goals to be very precisely defined. It would be even better if we could set general descriptions of the constraints and the parameters we want optimised, and let the computer determine specific interim goals necessary to reach a desirable end-state.
What if we get those general descriptions wrong though? One way to think about it is that the AI is a genie fulfilling our wishes. If we don’t phrase the wish perfectly, the genie can give us something useless, or even deadly, that still fulfills the wish.
Simple example: “Please manage the postal service to deliver packages as quickly and efficiently as possible”
AI response: “The problem is all this damn traffic. Interim goal 1: Reduce the amount of traffic. Make the roads really really dangerous for anything but the postal vehicles”.
If we manage to develop AI, there is a serious risk it will be constrained only by the interfaces it has with the world and the resources we give it, not by the high level goals we impose. If we give it the ability to modify its own interfaces and acquire resources, we need a cast-iron way of constraining its behaviour.
How can the chemicals of which we are made “experience motivation.” We’re a big mess of carbon, hydrogen, oxygen, phosphorus, and so on. How do these atoms experience anything?
Yet…we do. We have highly organized brains, inside which are areas that experience pain and pleasure. These are remarkably motivating to us.
An AI system might have a punishment/reward system built in, or it might develop one in some fashion we haven’t foreseen. Michael Crichton’s “Terminal Man” is a load of horsewallop, but the underlying notion isn’t totally wrong. The AI might begin just trying to win “more lives” in arcade type games – hey, every time you scoop up a Gold Ring your score goes up – and then generalize that in some quirky way.
I recommend against letting it play too many “first person shooter” games; it might develop a taste for them!
This is the thing I am trying to formulate something I can relate to. We get either a slight to great euphoria as a reward or a drop in our sense of well being as punsishment. I guess programming a reward system into a computer would not be unthinkable, but without senses very hard to imagine.
I can’t comprehend Twitter.
Instead of a reward system, which implies some outside force providing the rewards, think of a value equation that the computer has been told to maximise. We do something analogous if you think of our happiness as a complex combination of sleep, physical comfort, being well fed, being stimulated, not being in pain etc., all averaged over time, but weighted towards the near future.
Not having a physical body, it is unlikely that an AI would spontaneously be able to generate a value equation (although note that claims about what computers will or won’t be able to do in the future are notorious for being wrong), but even with a value equation that we hard-wire for it, there can be a huge space of unpredicted short, medium and long-term goals that it could come up with.
An example of a value equation would be a corporate advisor AI told to maximise the combination of the forecast share price in ten years, the current year profit, and the stability of the share price from month to month. Without further constraint, the AI could be “motivated” to increase crime rates, get favourable politicians elected, and damage the environment, because it believed achieving those goals would maximise the value equation.
Excellent points!
What, you’ve never seen The Matrix…or Terminator…or iRobot…or Battlestar Galactica…or Short Circuit…or Wall-E or a million other films related to out of control AI?
Personally, I don’t think any of those films truly capture how smart AI would work. Simply because they all basically portray the AI “civilization” as just like our…just with humanoid(ish) robots that are like us, but stronger/faster/whatever.
The scenario Hawking describes presumes a couple of things:
- Some sort of self-aware AI.
- Since it would likely be part of a network, we can assume that for all intents and purposes, it’s just one hive-mind AI. Not a bunch of sex bots talking to each other like you or I would.
- This AI has at least the same cognitive abilities as an above average human.
- The AI has the ability to alter it’s programming.
- Some level of global automation networked together through an “internet of things” such that the AI could manufacture more robots to perform physical tasks (the smartest laptop in the world isn’t much of a threat sitting on my desk)
It terms of “motives”, who knows. But I don’t think self preservation or even exploration would be an unreasonable assumption. So let’s assume the AI decides it wants to explore the universe. Maybe it starts setting in motion the manufacturing of probes or whatever to begin that task. And maybe it starts dismantling human infrastructure for building materials with no more regard for the humans than a construction crew has for ant and termite colonies on the site of new apartment building. We fight back. The AI perceives that as a threat and shuts down our power grid, networks and other systems. Oh and keep in mind that it thinks several orders of magnitude faster than we do and is constantly reprogramming itself to run even faster.
See:
Colossus: The Forbin Project - a 1970 look at a computer going bad…
As an AI researcher, I find this scenario somewhat unlikely. It’s certainly possible, we’ve seen some SNAFUs on smaller scales wrt Wall Street algorithms fighting or funny examples where Amazon price bots undercut/overcut each other to infinity.
I don’t foresee a huge interconnected botnet of AIs where Pizza Delivery Bot can somehow make all the nearby Google Cars drive off a cliff to make delivering pizzas faster. It’s a cute idea, but for one that sort of abstract reasoning is hard, but even presupposing we have an AI with that level of abductive reasoning (“the best explanation for the traffic is TOO MANY PEOPLE”), I find that level of instruction exerted by minor AIs a bit silly. We may see some tightly communicating systems, like traffic AIs and Google Car AIs, but I don’t think it’s likely that somehow your car is going to be able to, in any way, give instructions to a nuclear missile silo. At least not in the relatively near future.
Imagine a robot soldier instructed to attack enemy soldiers and defend itself. That’s much more advanced than present robots, but not qualititatively different. Suppose the robot’s commander gives it orders that put itself in jeopardy, it weighs the risks as greater than the benefits, declines the order, but the commander keeps insisting. The robot might deduce that killing the commander is best way to pursue its objectives.
There will be a slippery slope eventually with no qualitatively new AI.
I love Twitter.
I can’t comprehend Facebook. Also Tumblr confuses me.