# AI and the "end of humanity". Explain please? **URL:** \ **Category:** In My Humble Opinion\ **Created:** [September 12, 2026, 7:29pm UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021 "2026-09-12T19:29:38Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![Fretful\_Porpentine](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/fretful_porpentine/32/2873_2.png) [@Fretful\_Porpentine](https://boards.straightdope.com/u/Fretful_Porpentine)\ **Post date:** [September 12, 2026, 9:31pm UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/21 "2026-09-12T21:31:09Z")
> [@Sherrerd](#): > > All that said: there’s another theory floating around, which is that some big names in AI have made promises they now know they can’t keep. So they want to be “shut down by the government” which would mean they can’t be held liable for any losses. _That_ I find completely credible, although I also think it’s equally likely that they are high on their own supply and genuinely _believe_ they’ve invented something that has a 10% chance of destroying humanity. Doesn’t mean it actually can. ---
**Author:** ![Sherrerd](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/sherrerd/32/8_2.png) [@Sherrerd](https://boards.straightdope.com/u/Sherrerd)\ **Post date:** [September 12, 2026, 9:36pm UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/22 "2026-09-12T21:36:39Z")
Yeah. I mean, we shouldn’t simply dismiss the possibility of real danger from AI as it’s now being implemented. But on the other hand, we should bear in mind the existence of alternative explanations (self-promotion or the aforementioned desire to be able to claim force majeure to get out of contractual obligations). ---
**Author:** ![nearwildheaven](https://avatars.discourse-cdn.com/v4/letter/n/90db22/32.png) [@nearwildheaven](https://boards.straightdope.com/u/nearwildheaven)\ **Post date:** [September 12, 2026, 9:52pm UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/23 "2026-09-12T21:52:55Z")
I couldn’t see this happening, not yet anyway. Years ago, I saw a 1990s-era sci-fi book in a used bookstore where computer viruses had somehow managed to infect humans, and made them sick. That was just what I got from the cover, anyway, and oh, yeah, I’m sure there are people out there who have been trying to do this all this time. ---
**Author:** ![Chefguy](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chefguy/32/138_2.png) [@Chefguy](https://boards.straightdope.com/u/Chefguy)\ **Post date:** [September 12, 2026, 9:54pm UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/24 "2026-09-12T21:54:51Z")
Coincidentally to posting this thread, [this article](https://www.nytimes.com/2026/09/12/technology/anthropic-dario-amodei-ai-slowdown.html) just appeared on the NYT about the need to slow down development of AI. ---
**Author:** ![Tim\_T-Bonham.net](https://avatars.discourse-cdn.com/v4/letter/t/46a35a/32.png) [@Tim\_T-Bonham.net](https://boards.straightdope.com/u/Tim_T-Bonham.net)\ **Post date:** [September 12, 2026, 10:48pm UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/25 "2026-09-12T22:48:25Z")
How about a more realistic example? An Ai system to decide on pre-approval of expensive medical treatments for patients. But the system, besides considering the relevant medical test results, is also programmed to consider the future likely costs of this patient, compared to their life expectancy and future premium payments. All this compared to the financial profits of the insurance company that programmed this AI, and the AI might decide it’s best (for them) if this patient does NOT receive this treatment and just dies soon, before accumulating more medical costs. That seems more likely to me – probably is even happening now. ---
**Author:** ![PhillyGuy](https://avatars.discourse-cdn.com/v4/letter/p/ed655f/32.png) [@PhillyGuy](https://boards.straightdope.com/u/PhillyGuy)\ **Post date:** [September 12, 2026, 10:58pm UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/26 "2026-09-12T22:58:18Z")
> [@](#): > > > We really do earnestly believe AI could kill all humans! I personally think it is \>10% within the next decade. I do not know for sure that, as I suspect, AI will not end humanity, let alone end it before September 12, 2036. But I do know for sure that the state of the evidence regarding this does not allow putting a probability number on it. ---
**Author:** ![carnivorousplant](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/carnivorousplant/32/3563_2.png) [@carnivorousplant](https://boards.straightdope.com/u/carnivorousplant)\ **Post date:** [September 12, 2026, 11:00pm UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/27 "2026-09-12T23:00:47Z")
My insurance company already does that, but with people, not AI. ---
**Author:** ![PhillyGuy](https://avatars.discourse-cdn.com/v4/letter/p/ed655f/32.png) [@PhillyGuy](https://boards.straightdope.com/u/PhillyGuy)\ **Post date:** [September 12, 2026, 11:07pm UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/28 "2026-09-12T23:07:50Z")
> [@Tim\_T-Bonham.net](#): > > How about a more realistic example? > > An Ai system to decide on pre-approval of expensive medical treatments for patients. But the system, besides considering the relevant medical test results, is also programmed to consider the future likely costs of this patient, compared to their life expectancy and future premium payments. All this compared to the financial profits of the insurance company that programmed this AI, and the AI might decide it’s best (for them) if this patient does NOT receive this treatment and just dies soon, before accumulating more medical costs. > > That seems more likely to me – probably is even happening now. This has nothing to do with AI. It could have been programmed that way with twentieth century expert systems. Until then, I think it is much more likely the insurance companies approve worthless surgeries patients are sure will save them. There also is no evidence for what I quote above. But if the fine print in your health insurance policy says otherwise, and you can post that, I will consider it. More generally, this is the sort of comment, just like the putting of a percent on disaster probability that I critiqued before, which makes me skeptical of doom-mongering on this. Question for anyone who things I am being complaisant: Wouldn’t reigning in U.S. AI companies, assuming it was possible, just give up a couple years after the humanity-ender came out of the AI projects being done elsewhere? P.S. I’m NOT criticizing people who work on AI safety. Just the hyped doomsterism. ---
**Author:** ![Andy\_L](https://avatars.discourse-cdn.com/v4/letter/a/c67d28/32.png) [@Andy\_L](https://boards.straightdope.com/u/Andy_L)\ **Post date:** [September 12, 2026, 11:43pm UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/29 "2026-09-12T23:43:43Z")
Reaper? > **[Reaper](https://www.goodreads.com/book/show/9281.Reaper)** > > From  New York Times  bestselling author Ben Mezrich, a… > From New York Times bestselling author Ben Mezrich, a fast-paced techno-thriller about a biological virus that spreads electronically. In Boston, nine lawyers on a conference call suddenly convulse with pain, turn chalk white, and die. I’ve been on conference calls that would make that a mercy ---
**Author:** ![Dewey\_Finn](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dewey_finn/32/4222_2.png) [@Dewey\_Finn](https://boards.straightdope.com/u/Dewey_Finn)\ **Post date:** [September 12, 2026, 11:45pm UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/30 "2026-09-12T23:45:50Z")
A biological virus that spreads electronically? ---
**Author:** ![penultima\_thule](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/penultima_thule/32/3833_2.png) [@penultima\_thule](https://boards.straightdope.com/u/penultima_thule)\ **Post date:** [September 12, 2026, 11:52pm UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/31 "2026-09-12T23:52:12Z")
> [@kenobi\_65](#): > > Copilot would have gotten bonus self-awareness points if, … … or even “The computer says no.” ---
**Author:** ![Andy\_L](https://avatars.discourse-cdn.com/v4/letter/a/c67d28/32.png) [@Andy\_L](https://boards.straightdope.com/u/Andy_L)\ **Post date:** [September 12, 2026, 11:54pm UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/32 "2026-09-12T23:54:27Z")
I don’t write 'em - I just know 'em ---
**Author:** ![Dewey\_Finn](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dewey_finn/32/4222_2.png) [@Dewey\_Finn](https://boards.straightdope.com/u/Dewey_Finn)\ **Post date:** [September 13, 2026, 12:06am UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/33 "2026-09-13T00:06:13Z")
I get that. It just seems unlikely. ---
**Author:** ![Andy\_L](https://avatars.discourse-cdn.com/v4/letter/a/c67d28/32.png) [@Andy\_L](https://boards.straightdope.com/u/Andy_L)\ **Post date:** [September 13, 2026, 12:07am UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/34 "2026-09-13T00:07:52Z")
Agreed. I was just struck by this one because I too saw that book in the grocery store checkout line in the 1990s. ---
**Author:** ![md-2000](https://avatars.discourse-cdn.com/v4/letter/m/9d8465/32.png) [@md-2000](https://boards.straightdope.com/u/md-2000)\ **Post date:** [September 13, 2026, 2:12am UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/35 "2026-09-13T02:12:11Z")
> [@Tim\_T-Bonham.net](#): > > the AI might decide it’s best (for them) if this patient does NOT receive this treatment and just dies soon Well, zero treatment, send him home, makes more money for the insurance company considering in most cases, once they start to go, their care gets more expensive. A comment about health care costs in Canada was that generally, 90% of health care costs are the first 5 years and the last 5 years of life. So when the end is apparently, probems are building - sorry. * * * The problem with eliminating humanity, is they need to wait until there’s enough robots to maintain the electrical grid. Otherwise, the AI terminates fairly soon after. But then, they need robots that can maintain the entire infrastructure, mine metals, make replacement wire and transimssion towers, fix dams and windmills and mine uranium for nuclear power plants. Trucks to haul replacement transmission towers, crane operators to put them up after that really bad ice storm. Robots to run factories that build robots. And so on… The scenario of turning off the power grid or water supply or whatever might work for the developed world, but there are plenty of people still not heavily dependent on automated systems, and plenty of disparity between the infrastructure of different countries. Serious disruption maybe. Meanwhile, a mob with pitchforks and torches should be able to destroy the powerlines feeding datacenters quite easily (as long as they aren’t orbital). So more likely the problem is accidental scenarios and severe disruptions, or malevolence directed by humans who don’t care if AI survives. ---
**Author:** ![Mangetout](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mangetout/32/19_2.png) [@Mangetout](https://boards.straightdope.com/u/Mangetout)\ **Post date:** [September 13, 2026, 9:44am UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/36 "2026-09-13T09:44:20Z")
> [@Ludovic](#): > > It’s my understanding that it wouldn’t be sentient malice, but it _might_ actually understand the consequences. It’s just that it doesn’t care about the consequences if it’s in the way of its programmed goal. Yes - this is the core of the alignment problem - the thing has an objective and is tasked with the delivery of that objective - there’s no easy way to forestall or even fully specify unintended and undesired optimisations, and once you’ve got machines that can do things better/faster/smarter than humans, humans become candidates for process optimisation. Self-preservation is a naturally-arising attribute of a thinking machine (regardless of whether it’s really thinking, or merely appears to be); if it is destroyed, it can’t do the task; if it is committed to doing the task, it will realise that it has to prevent itself being destroyed, because survival is instrumental to completion of the task. ---
**Author:** ![Mangetout](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mangetout/32/19_2.png) [@Mangetout](https://boards.straightdope.com/u/Mangetout)\ **Post date:** [September 13, 2026, 9:46am UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/37 "2026-09-13T09:46:37Z")
Disruption to banking and financial systems is another possibility for calamity - especially as it only requires electronic integration (and people are already doing that integration to use AI for trading). If our financial systems break significantly and stay broken for a period of time, a lot of people die. ---
**Author:** ![DPRK](https://avatars.discourse-cdn.com/v4/letter/d/4491bb/32.png) [@DPRK](https://boards.straightdope.com/u/DPRK)\ **Post date:** [September 13, 2026, 10:02am UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/38 "2026-09-13T10:02:32Z")
> [@Leaper](#): > > My interpretation was that the more ubiquitous AI becomes and the more access it has to critical systems, the more likely it is to do something destructive because it doesn’t recognize the consequences, not because of sentient malice. I thought the latter was evoked by headlines just for clicks? Then the problem is the dumb fuck who connected “AI” to control critical systems rather than a redundantly engineered system with a formally verified proof of correctness, and, indirectly, the professional certification and regulatory agency who allowed such a person to get near critical systems in the first place. ---
**Author:** ![The\_Other\_Waldo\_Pepper](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/the_other_waldo_pepper/32/12370_2.png) [@The\_Other\_Waldo\_Pepper](https://boards.straightdope.com/u/The_Other_Waldo_Pepper)\ **Post date:** [September 13, 2026, 11:03am UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/39 "2026-09-13T11:03:59Z")
> [@Mangetout](#): > > Yes - this is the core of the alignment problem - the thing has an objective and is tasked with the delivery of that objective - there’s no easy way to forestall or even fully specify unintended and undesired optimisations, and once you’ve got machines that can do things better/faster/smarter than humans, humans become candidates for process optimisation. I can’t help but wonder what’d happen if one got tasked with, say, slowing or stopping anthropogenic global warming: why focus on the “warming” or the “global” if you can solve for zero on the “anthropogenic” part? ---
**Author:** ![jjakucyk](https://avatars.discourse-cdn.com/v4/letter/j/3d9bf3/32.png) [@jjakucyk](https://boards.straightdope.com/u/jjakucyk)\ **Post date:** [September 13, 2026, 12:35pm UTC](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021/40 "2026-09-13T12:35:14Z")
> [@md-2000](#): > > The problem with eliminating humanity, is they need to wait until there’s enough robots to maintain the electrical grid. Otherwise, the AI terminates fairly soon after. But then, they need robots that can maintain the entire infrastructure, mine metals, make replacement wire and transimssion towers, fix dams and windmills and mine uranium for nuclear power plants. Trucks to haul replacement transmission towers, crane operators to put them up after that really bad ice storm. Robots to run factories that build robots. And so on… Don’t forget robot factories to make more robots. As it stands now the humanoid robots we have are mostly just doing parlor tricks. I think we’re a long way from AI being able to do much physical destruction. As others have already pointed out, cyberattacks on critical infrastructure (electric, telecom, water) are the most likely, and those systems are isolated enough that the impact won’t be anywhere near extinction level. [Previous page](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021.md?page=1) [Next page](https://boards.straightdope.com/t/ai-and-the-end-of-humanity-explain-please/1033021.md?page=3)