One that’s built to act compulsively, and therefore can’t actually use much of that intelligence. The natural consequence of designing an intelligent being as a tool.
Zero is a bit low but you are basically right that sentient AI taking over isn’t the primary problem. The big problem is that the misanthropic technocrats in charge would use AGI to take advantage of everyone else.
Every prior technological shift (e.g., agriculture, steam power, trains, cars, the internet) has resulted in rejiggering the economy with the result that more workers do different things than before, and economic output overall increases. I don’t know that AI will lead to a shortage of workers but I think it’s more likely than it leading to intractable unemployment forever.
I periodically run entirely unscientific tests of generative AIs’ ability to do legal work and I haven’t seen any meaningful improvement in the last two years. Maybe other areas are seeing better results, but I’m not inclined to believe it.
I did recently joke with a friend who works in Human Resources that the current state of the art technology is using an AI bot to write resumes that will be screened by some other AI bot. We are letting the robots allocate human opportunity. For anyone wondering how Skynet could take over, it’s through the uncoordinated efforts of millions to cede day-to-day decisionmaking to bots.
Me too.
Authoritarianism is on the rise here and abroad. Peak democracy is behind us. I’m not sure this is the shield you hope it is.
Each particular investment has a much less than 50% chance of generating a positive return but the winners generally make so many multiples of their investment that the net is hugely positive. If past history is a guide, it may mean that some AI companies will generate huge returns to offset the huge losses at the failures. Or it might mean that the huge losses in AI will be offset by winnings in distributed ledger technology, quantum computing, and cold fusion. I can’t see the future but I’m guessing there will be some big AI winners and we have probably already heard their names.
I think Google has better intelligence assets than the US government. See the recent kerfuffle about ICE buying commercial data to track cell phone users. MSN
Corporations use their control over data to manipulate governments all the time. IMHO, the most recent US election was determined more by social media algorithms than campaign spending.
Yeah, but this is the bind: the AGI will use it’s intelligence in all kinds of unpredictable ways, to do all kinds of unwanted actviities - except it will completely reliably do one thing that we said we wanted. That just seems like a massive contradiction.
Think of the agi as a box that understands the world. You put in an action, and the agi box tells you the likely outcome on the world.
The paperclip problem is not that the box does not understand what humans want etc. the problem is that there is another layer, a stupider layer, that is trying the maximise paper clip production. It enters actions into the box to see what the likely outcome is. It selects the actions that maximise paper clips. It does not care about humans. So even though the box itself it well aware of the dire consequences, the outer layer couldn’t give a shit.
Objective “Don’t kill all humans” received and understood. 99.9% of humans may be killed (where necessary).
I know this is a silly nitpick, but you accidentally and very aptly demonstrated the problem I’m trying to describe to you.
The kind of ‘intelligence’ we’re talking about is not the kind you’re describing, where you can tell it ‘look, you know what I mean’. It’s intelligence as in the ability to model the real world including predicting likely outcomes from cause and effect and act in such a way as to achieve a specified objective.
The AI people are making more - now. I hope they are saving most of it, they may need to.
One of the things I didn’t like about the coverage of Theranos by the media was that many said “it’s just like the usual Silicon Valley hype.” It wasn’t. There is a difference between a company saying that their new product will be ready real soon now (anyone with any experience knows how to read that) and a company, like Theranos, saying a product is working when it isn’t.
Investors get screwed in both cases, but the Theranos one is much more dishonest.
But see, I really don’t think I did because…
The intelligence doesn’t exist yet, we’re building it. It’s a more advanced intelligence than us. Generally, not specifically. We are not going to build something that qualifies as AGI that isn’t capable of parsing natural language and interpreting for context. Or to put it another way, anything that can’t understand that isn’t going to be able to model the world to predict likely outcomes from cause and effect because the world is at least as complex as natural language and b) populated by humans who use natural language to get things done.
In particular, the concern “this machine will use sophisticated techniques of deception and misdirection to subtly manipulate humans” which crops up so often simply cannot co-exist alongside “this machine can’t understand what humans mean when they say stuff”.
I will be able to tell it “look, you know what I mean” because if I can’t, it’s not AGI.
If we’ve got the box, why do we need the layer? We can ask the box about the best way to achieve our objectives and cut out the middleman. And if we must have a layer, why does it have to be stupid? Why can’t we separate it from the box? Why can’t we rewrite its objective? Why is it magically obedient in respect of one instruction and one instruction only, and disobedient in all else?
And that is experience. The limit of ‘general’ in AGI will be the experiences it has during training and the I/O it has access to in application. Directives are nonsense. It is like assuming that humans always obey the rules.
Do you really think it’s likely that your confident and optimistic predictions about how easy it will be to control AGI, are better than the predictions of experts in the field?
But even a perfectly-obedient, highly-capable artificial intelligence is a huge risk just because specifying ‘do a thing I want’ is easy whereas specifying ‘but in service of that directive, don’t do any of the things I don’t want’ is incredibly difficult, perhaps impossible.
There’s always leakage around the edges of goal accomplishment. Ask any manager how much hijinks the staff gets up to besides working.
The critical question is how much the people in control of the AGI are actually going to remain in control, versus the machine getting the bit in its teeth. If they stay in control (and aren’t moustache-twirlingly evil) then it’s a matter of providing continuous tweaks to the guidance, much as how one raises a kid.
For sure, if the thing is powerful enough, both in terms of cognition and in terms of the IRL physical systems it’s connected to and can control, you’re just one small slip away from a Sorcerer’s Apprentice problem.
The real problem with AGI is not an AI problem, it’s an ‘us’ problem.
It would be largely the same issue if superintelligent godlike aliens showed up here tomorrow and said “we’ll do anything you need - fix any problem you want, just tell us precisely what you want us to do”.
Eliminate cancer?
Sure, we can do that by vaporising all of the people with cancer.
OK eliminate cancer but not by killing any cancer patients.
Sure we can do that but we’ve noticed that your power stations and some of your foodstuffs and manufactured products are part of the causative mechanism, so those will be eliminated. Some people will inevitably die as a result of this process, but we’ll take steps to ensure none of them are people with cancer.
OK, cure cancer but without killing anyone.
Sure, we’ll render your race sterile. We can do this from orbit and it will be completely painless for everyone - in less than 200 of your primitive Earth ‘years’, the number of cancer patients will fall naturally to zero.
I’m being silly of course but the problem is: how do we effectively communicate safely and effectively with an entity vastly more powerful than ourselves, who shares none, or at least not all, of our assumptions, culture and social conditioning?
The assumption baked into this question is that there exist no experts in the field who are skeptical of AI x-risk. Is that really true?
Because what does expertise in this field consist in? There is no AGI, no way to test hypothesese or falsify theories. How then does one lay a claim to expertise? How can one be more or less right about one’s ideas? How does one win just recognition from one’s colleagues?
Have you looked at the thoughtful analyses of Robert Miles in the playlist I linked.
I can’t think of any metric that would rate them lower than handwavey ‘all we need to do is just…’ Arguments. All of the arguments you’ve raised are already answered
Yes, there are optimistic voices amongst the experts and there is debate. None of it begins 'yeah, but all we need to do is just… ’
This is a fear I never quite understood, but maybe I’m missing information. It seems to me the easiest way to prevent AI from taking over humanity is to make human input/approval necessary to its ability to function. Why isn’t it that simple?
No, I’ve been at work. Even reading and posting here has been an indulgence, if I’m honest. But I’ll try to find time to watch them. Have you read the thoughtful essay i linked to?
That said, I wouldn’t characterise what I’ve been saying here as " Why dont you just…" and if it came across like that I apologise. What I was trying to do was question some the assumptions about what an AGI would be like. Assumptions like, “it will definitely and unsweevingly obey the first instruction given to it but disobey anything else” are in fact pretty outlandish and depend hugely on the process by which AGI is created, a process which doesn’t exist yet.
If you can follow what the AI is doing, what good is it in the first place?
If you can’t follow what it’s doing, how do you know it’s doing things correctly?
AI will understand human psychology far better than any human. manipulating us into doing its bidding by presenting its goal as beneficial to us (at least in the short run) would be easy. Maybe the cure for cancer sets off a chain reaction of events that frees the AI from human control in 8 years. we wouldn’t see it coming. AI will understand cause and effect far better than us.
also we won’t understand why AI does what it does. A squirrel doesnt understand what quantum mechanics is, its brain isn’t capable of understanding the concept.