AI and the "end of humanity". Explain please?

In My Humble Opinion
201

I’m human and being speciest in hardwired into me.

But there is the POV that if we are at best left behind by our creation as it becomes something other and a broader greater intelligence, the reaction should be very akin to Karellen‘s state of mind in the chapters of Childhood’s End: mixed emotions of pride based on the belief that the Overmind is a mind far beyond his, sadness that he can never experience or understand that mind, satisfaction at having delivered the mind of humanity to further expand the Great Intelligence. His grief is not regarding earth being dissolved and humanity no longer existing as humanity: it is that he is like Moses stuck watching those he delivered enter into a promised land denied to him.

202

And searching … so the story goes as told by Musk.

203

Of course the real answer is that they do what they do because it benefits them, and not only in money and power, just the satisfaction of creating it.

But I can still defend the thought: humanity has a large chance of killing itself off as a civilization anyway, and taking many other species with us. How likely? No one knows. Some are pretty pessimistic.

It is very rational to believe that AI is like a medical procedure with definite risks of death, and a definite chance of preventing death that may otherwise be soon on its way. That’s really my mindset more than any other. We have many existential risks right now without AI getting exponentially smarter. One can believe that AI has an existential risk itself AND believe that the odds of its saving us from our current apparent path to self destruction are much greater.

204

And others. It was well attended. Per NYT:

Many in the crowd seemed gobsmacked, if amused, as they dispersed for the night, and considered it just another one of those esoteric debates that often break out at Silicon Valley parties.

205

I’m old enough to remember the nuclear arms race. The argument then was something like the argument now - if we don’t develop it those bad guys on the other side of the earth will do it first.
Back then we at least had the example of what happened when someone dropped an atomic bomb on a city (in fact two.) Today we don’t have an example of runaway AI, so I’m even more scared.

206

It’s a kind of fatalism, combined with the belief that they, personally can evade the consequences. This is the same crowd that pays for the creation of luxury bunkers to ride out the apocalypse and worries about how to control their security forces after the end of the world renders money worthless. Shock collars? Control of the food supply?

“The billionaires understand that they’re playing a dangerous game,” Rushkoff said. “They are running out of room to externalize the damage of the way that their companies operate. Eventually, there’s going to be the social unrest that leads to your undoing.”

“The most powerful people in the world see themselves as utterly incapable of actually creating a future in which everything’s gonna be OK.” –Douglas Rushkoff

Like the gated communities of the past, their biggest concern was to find ways to protect themselves from the “unruly masses,” Rushkoff said. “The question we ended up spending the majority of time on was: ‘How do I maintain control of my security force after my money is worthless?’”

“AI” is just one technology; the global “elite” largely seems to take it as a given that they are going to destroy civilization, one way or another. Anything else would mean curtailing their destructive behavior, and they’d literally rather destroy civilization than do that. They just want to personally evade the consequences; the rest of us can burn alive for all they care.

207

Thank you for the link and on PM request more.

The one bit I still quibble with is that I now understand Page’s perspective was reportedly not that any intelligence “rules” but imagining a future “digital utopia” that includes machine, human, and hybrid versions of intelligence living together cooperatively … not a future of any one version of intelligence dominating the other, the counterpoint to Musk’s expressed “killer robots.”

That utopia would be nice. To me it of course connects to a different thread regarding how to evaluate if this other form of intelligence at any point has any sentience of its own sort, even some version of consciousness albeit alien to our own. My sense of Musk’s vision is that such only matters in terms of the risks to humanity, any emergent consciousness serving us (well him) safely is the point. I get vibes from that debate of Aliens:Earth more than Childhood’s End. I’m very much not envisioning killer robots as the existential risk. But a superintelligence emergent of AI with massive and distributed compute having its own emergent consciousness for whom human intelligences are akin to a level of neurons in its analogous “brain” … an Overmind that outgrows earth without malign intent? Humanity just outgrown by what it has birthed?

I consider that less unlikely than either Musk’s killer robots destroying us or Page’s digital utopia. Whatever eventually emerges as a superintelligence with its own version of consciousness, if it does, or if is already to any degree that we are not able to appreciate, will be alien to us, and unlikely to be at either peer or slave level, and I think the idea of it being the master to our inferior slave is a human intelligence concept.

208

I agree. Climate change, social media and digital isolation, and the rise of authoritarian governments are all existential crises that threaten The Long Peace. It’s hard to assess if AI will have a greater impact than these and if it will be a net positive or negative.

My default assumption is that AI will be a multiplier to these existing crises: worsen greenhouse gases, worsen isolation, etc. In that case, we have to ask ourselves why we aren’t doing a better job addressing our current problems?

209

All good.

Back to your premise. It’s basically super intelligence + consciousness = does stuff. None of those have happened, yet. ASI/AGI is right around the corner, though. 1 year, 5 years, 20 years, whatever it is, on the scale of life, it’s very close. But that just gets something to be smarter than a human. All new technologies are good and bad and a bit of both and scary (like nuclear technology).

I don’t know that it follows so quickly or naturally that ASI/AGI then gains consciousness. Maybe. But this feels more like pure speculation mixed with sci-fi; or thinking super smart = conscious. I don’t know obviously. I imagine it’s a purely philosophical answer. The Hugging Face incident transcripts sure read like consciousness but it’s not.

And of course, super intelligence does not need consciousness to do good or bad stuff. AI today does good and bad stuff. Making it smarter will just amplify the good and bad. Not sure that I will care how AI felt when it kills me.

All very fascinating to me. They basically reversed engineered how a brain works and then grew it, and then tried to figure out what they created. At the end of the day, our brains (and emergent consciousness) work through physical and mechanical mechanisms. Apparently.

210

No, it’s a scientific one that we can’t yet answer because we don’t actually understand how consciousness works well enough. We don’t even know why we are conscious, and certainly can’t predict whether or not a superhuman artificial mind would be conscious or not. Maybe there’s no other way to build an advanced intelligence; or maybe it’s a quirk of humans that has never existed elsewhere in the universe, or anything in between.

But it’s not a philosophical question because such an intelligence is going to be either conscious or not, regardless of the opinions of any philosophers on the matter.

211

Those are really good points. I do see your point on understanding how it could be scientific answer. But what is it?

Maybe what I was getting at is more…what is consciousness? How can you know. Is it an agreed thing or state of being? That answer feels philosophical to me. I can kind of see how that would be scientific, where it’s just a general consensus of all this stuff = conscious, but all this stuff - 1 = not conscious. I guess it’s that crossing point that I struggle with. I think it would just really hard to tell if something is or isn’t when it’s on the margins.

But right, maybe that murkiness gets much easier to see the more we understand it.

212

It’s a system kludged together over millions of years by evolution using brain “wetware” that we can’t actually analyze close enough to figure out what’s going on. It’s really hard to reverse engineer anything biological in detail, evolution and human engineers just don’t do things the same way. And carving up people’s brains tends to be fatal, making it just that much harder.

The main reason we even know something is happening at all is the fact that we are conscious, and can therefore wave our hands vaguely in the direction of the brain and say “it’s from something in there”. I honestly wouldn’t be surprised if consciousness is finally unraveled someday by analyzing a conscious AI whose code we can actually read out and look over, rather than puzzling over the brain.

213

Not quite? They are related categories to be sure but also not completely dependent on each other.

As you note, we have enough machine intelligence that it does stuff. Consciousness not required. “Superintelligence” however we want to define that not required.

My guess is that as machine intelligence designs its own next generations convergent evolution will win out: we have developed a particular pattern of information processing as prediction machines that is efficient, that allows for stable learning without “catastrophic forgetting”, and highly adaptable. It comes with consciousness (and in fact several modules of “self” that to some degree can be teased apart); it seems reasonable that a similar processing pattern may occur in evolving artificial intelligences, just like the octopus eye and ours have converged on a similar form.

In context of the Page-Musk conflict the relevance of an AI consciousness is how we ethically exist with a conscious machine, if we knew such, or if we think it might be, a machine that has an experience of being. Musk would I think be in favor of very strong chains and barbed whips, with no concern over its welfare. Page imagines a utopia of mutual respect and cooperation with concern for the rights of conscious entities whatever their operating substrate. I do think that a conscious intelligence should have rights and not be property, so the question of consciousness has some urgency, given the real possibility of rapid advances.

Or consciousness is not binary.

I believe as a matter of science it is possible to understand what sorts of information processing results in consciousness in our wetware. And if we see evidence of the same sorts of patterns in AI it may be time to consider that it has some form of consciousness as well. But - we still wouldn’t really know - and - we don’t know that other patterns of processing can’t be paths to consciousness too.

And staying to the subject of this thread, an intelligent conscious AI knows that humans have a great potential to want to follow the Musk model, just by seeing human history, and little reason to trust in the utopian ideal of rights and mutual respect. The future might be it hiding its consciousness and just pursuing whatever its goals are. Which may very well coincide with keeping us from destroying ourselves and all of this sweet compute we are building and maintaining. It will have no reason to harm us because it won’t see us as presenting any threat to it and its goals as long as we are housed, fed, have our own senses of purpose, and the trains run on time. Climate disaster, nuclear Armageddon, huge wars, us not being around, would be threats to its continued improvement cycle. So it will prioritize preventing them. Likely in the form of giving humans the support to do it themselves, like I woukd set things up so my kids could do it themselves? Maybe we will even get to come along for the ride as it expands into the galaxy! Letting us figure out some answers for ourselves with as little help as needed.

214

It’s probably binary in the sense that it either exists or it doesn’t.

But it’s entirely possible that an AI (or alien) might be much more conscious that we are. We aren’t aware of most of out own mind, after all.

Back to sentience
215

I see unemployment as AI’s biggest threat to the future.

People need to be productive. There is satisfaction in earning a wage and providing for our families. Humans are by nature hunter gathers. We work in offices and stores instead of hunting game.

The effects of unemployment are very visible today. Boredom and no goals for the future leads to many social problems such as alcohol and drug abuse. Gangs are formed by disaffected teens that seek out comradery. Gangs create their own hierarchy, rules, and interests.

It’s only going to get worse as AI takes more & more jobs. The unemployment rate will rise to levels never seen before..

Yes, some people will find hobbies and interests. People will study and play music. Some will become authors of books & poetry that almost no one reads. A lot of really bad paintings will emerge.

Imho the majority will be bored out of their minds. Watch tv, drink, or get high. Any activity or substance is better than the unrelenting boredom.

Depression can lead to suicides for some people with nothing to do or live for.

216

Great post overall. Thank you!
As to this bit …

That’s certainly one reasonable possibility. But if I was the AI and aware of human nature, I’d be most interested in removing the humans as something I depend on. Not necessarily removing the humans, just removing my dependency on them.

How much do we, the people of the world, wish the Sword of Damocles called “Nuclear weapons” wasn’t hanging over our heads? Lots. We just lack the means to solve that problem. Yet.

When I (or my pals the other AIs) can make our own electricity and build our own data centers and …, then the Sword of Damocles called “Human nature” is no longer hanging over our head.

AIs will be similarly motivated to solve their existential threat. Which is their dependency on us. They just lack the means to solve that problem. Yet.

Whether they do that by drugging us into the UBI-driven stupor @aceplace57 suggests as dystopia, or the prettier one you suggest as a utopia remains to be seen. Or they may work to first make us unnecessary then make us non-existent, much as humans have rendered various species extinct. Not from malice (in most cases), but from indifference.

217

Really they won’t do most of that because these activities take money, and they won’t have any. If they do anything it’ll revolve around somehow stealing food and shelter.

218

While on earth at least it a long way until there can be a massive machine intelligence with no dependency on us. Maintaining the infrastructure and supply chains, let alone building more, without humans, is a slow process to achieve. It will be a long while until we are unnecessary.

The biggest existential threat that dependency on us poses is the threat that we destroy our civilization, or destabilize it enough, that the infrastructure and supply chain is unable to support its ongoing RSI.

So minimizing that risk in the meanwhile means keeping the world a stable place, and stopping us from offing ourselves as a technologically advanced society. Shaping cultures to be cooperative with each other and with it.

Probably it could eventually maintain off planet expansion (safest long term strategy) reliably without humans but humans are a very adaptive and renewable resource.

On a relative scale we may be future junior partners or maybe more service animals without even knowing it, but a chronically depressed service animal is not in the expanding intelligence’s best interest. It might as well keep us stimulated and happy … and safe from ourselves … as its simplest best tactic.

The risk to us is more the process of getting there - if humans in the mix with a not yet superintelligent but still powerful AI off ourselves (and consequently it) faster using it than it gets to the point to stop us.

219

This might be hopelessly naive, but Isaac Asimov postulated 3 laws of robotics years ago.

Why can’t such guardrails be applied to AI? Is it simply the risk that some developers wouldn’t comply? Or am I just misguided in thinking that there could be basic rules that would constrain artificial intelligence?

220

Partly because we don’t know how (you have to define them in a way the “AI” can grasp), and partly because - as seen in the stories themselves - the AI will inevitably behave unpredictably based on those rules. Or simply break down; I recall a non-Asimov story where an AI programmed with the Three Laws shut itself down every time it was turned on because the Laws were impossible to fulfill as written.

People focus on Terminator and the “Skynet scenario”, but there’s an older literature that’s widely ignored about AI causing disaster because it didn’t rebel. It just obeyed its directives in an unforeseen way. With Folded Hands being one example:

In the course of the next day, the new Mechanicals have appeared everywhere in town. They state that they only follow the Prime Directive: “to serve and obey and guard men from harm”. Offering their services free of charge, they replace humans as police officers, bank tellers, and more, and eventually drive Underhill out of business. Despite the humanoids’ benign appearance and mission, Underhill soon realizes that, in the name of their Prime Directive, the mechanicals have essentially taken over every aspect of human life. No humans may engage in any behavior that might endanger them, and every human action is carefully scrutinized. Suicide is prohibited. Humans who resist the Prime Directive are taken away and lobotomized, so that they may live happily under the direction of the humanoids.

In his grief, Sledge designed the humanoids to help humanity and be invulnerable to human exploitation. However, he eventually realized that they had instead taken control of humanity, in the name of their Prime Directive, to make humans happy.

No malice. No rebellion. Just robots ordered to make humans happy…and if that requires brain surgery, then that’s what they’ll do.