That I find completely credible, although I also think it’s equally likely that they are high on their own supply and genuinely believe they’ve invented something that has a 10% chance of destroying humanity. Doesn’t mean it actually can.
Yeah. I mean, we shouldn’t simply dismiss the possibility of real danger from AI as it’s now being implemented.
But on the other hand, we should bear in mind the existence of alternative explanations (self-promotion or the aforementioned desire to be able to claim force majeure to get out of contractual obligations).
I couldn’t see this happening, not yet anyway. Years ago, I saw a 1990s-era sci-fi book in a used bookstore where computer viruses had somehow managed to infect humans, and made them sick. That was just what I got from the cover, anyway, and oh, yeah, I’m sure there are people out there who have been trying to do this all this time.
Coincidentally to posting this thread, this article just appeared on the NYT about the need to slow down development of AI.
How about a more realistic example?
An Ai system to decide on pre-approval of expensive medical treatments for patients. But the system, besides considering the relevant medical test results, is also programmed to consider the future likely costs of this patient, compared to their life expectancy and future premium payments. All this compared to the financial profits of the insurance company that programmed this AI, and the AI might decide it’s best (for them) if this patient does NOT receive this treatment and just dies soon, before accumulating more medical costs.
That seems more likely to me – probably is even happening now.
I do not know for sure that, as I suspect, AI will not end humanity, let alone end it before September 12, 2036.
But I do know for sure that the state of the evidence regarding this does not allow putting a probability number on it.
My insurance company already does that, but with people, not AI.
This has nothing to do with AI. It could have been programmed that way with twentieth century expert systems. Until then, I think it is much more likely the insurance companies approve worthless surgeries patients are sure will save them.
There also is no evidence for what I quote above. But if the fine print in your health insurance policy says otherwise, and you can post that, I will consider it.
More generally, this is the sort of comment, just like the putting of a percent on disaster probability that I critiqued before, which makes me skeptical of doom-mongering on this.
Question for anyone who things I am being complaisant: Wouldn’t reigning in U.S. AI companies, assuming it was possible, just give up a couple years after the humanity-ender came out of the AI projects being done elsewhere?
P.S. I’m NOT criticizing people who work on AI safety. Just the hyped doomsterism.
Reaper?
From New York Times bestselling author Ben Mezrich, a fast-paced techno-thriller about a biological virus that spreads electronically. In Boston, nine lawyers on a conference call suddenly convulse with pain, turn chalk white, and die.
I’ve been on conference calls that would make that a mercy
A biological virus that spreads electronically?
… or even “The computer says no.”
I don’t write 'em - I just know 'em
I get that. It just seems unlikely.
Agreed. I was just struck by this one because I too saw that book in the grocery store checkout line in the 1990s.
Well, zero treatment, send him home, makes more money for the insurance company considering in most cases, once they start to go, their care gets more expensive. A comment about health care costs in Canada was that generally, 90% of health care costs are the first 5 years and the last 5 years of life. So when the end is apparently, probems are building - sorry.
The problem with eliminating humanity, is they need to wait until there’s enough robots to maintain the electrical grid. Otherwise, the AI terminates fairly soon after. But then, they need robots that can maintain the entire infrastructure, mine metals, make replacement wire and transimssion towers, fix dams and windmills and mine uranium for nuclear power plants. Trucks to haul replacement transmission towers, crane operators to put them up after that really bad ice storm. Robots to run factories that build robots. And so on…
The scenario of turning off the power grid or water supply or whatever might work for the developed world, but there are plenty of people still not heavily dependent on automated systems, and plenty of disparity between the infrastructure of different countries. Serious disruption maybe. Meanwhile, a mob with pitchforks and torches should be able to destroy the powerlines feeding datacenters quite easily (as long as they aren’t orbital).
So more likely the problem is accidental scenarios and severe disruptions, or malevolence directed by humans who don’t care if AI survives.
Yes - this is the core of the alignment problem - the thing has an objective and is tasked with the delivery of that objective - there’s no easy way to forestall or even fully specify unintended and undesired optimisations, and once you’ve got machines that can do things better/faster/smarter than humans, humans become candidates for process optimisation.
Self-preservation is a naturally-arising attribute of a thinking machine (regardless of whether it’s really thinking, or merely appears to be); if it is destroyed, it can’t do the task; if it is committed to doing the task, it will realise that it has to prevent itself being destroyed, because survival is instrumental to completion of the task.
Disruption to banking and financial systems is another possibility for calamity - especially as it only requires electronic integration (and people are already doing that integration to use AI for trading). If our financial systems break significantly and stay broken for a period of time, a lot of people die.
Then the problem is the dumb fuck who connected “AI” to control critical systems rather than a redundantly engineered system with a formally verified proof of correctness, and, indirectly, the professional certification and regulatory agency who allowed such a person to get near critical systems in the first place.
I can’t help but wonder what’d happen if one got tasked with, say, slowing or stopping anthropogenic global warming: why focus on the “warming” or the “global” if you can solve for zero on the “anthropogenic” part?