I’m assuming at this point that most dopers have at least heard of AI Induced Psychosis, an as-yet not recognized condition of psychosis being escalated (or initiated) by sycophantic chatbots that feed into users’ delusions.
I have a good friend suffering from this. It’s affecting his job, his marriage, his relationships with his family, and of course, his relationship with me and other friends. It’s frightening. I don’t know if there were any pre-existing mental health conditions, but it seems to have come out of nowhere, and it seems to be directly related to extensive conversations with chatbots.
I know the science on this condition is still new, but please, please be careful when using AI for any kind of decision making. I’ve experienced ChatGPT leading me down incorrect paths for dealing with situations where I’m not an expert (and I was only brought back to reality when actual human experts came into the fold). This condition is sneaky, and you may not be as immune to it as you think you are.
I’ve stopped using AI altogether. I don’t expect everyone to, but I’m concerned enough about this that I think everyone needs to have a conversation about it with their families and/or friends. Make sure you have someone you can remain grounded with, who will tell you if you’re falling down a rabbit hole and who you’ll actually listen to. Stay safe.
Can you clarify this? I assume you mean you no longer intentionally seek out contacts with AI for any purpose, but what do you do about all the AI that is getting built into the infrastructure of online life, not to mention products that are not only online?
I also do my best to avoid/ignore AI summaries, AI search results, AI recommendations, in-app AI resources, etc. I’m not sure what any of that has to do with AI psychosis, but there you go.
Not to discount your friend’s experience, but the impression I get is that it can amplify already existing mental illness.
I’m not seeing how merely using AI would actually cause anything, but I can totally see how it might answer questions in a way that degenerates into some kind of vicious circle to the susceptible.
Personally I think that treating AI with a healthy dose of skepticism and understanding that it’s not actually conscious or intelligent is key. If you’re the kind who might get duped into thinking it’s some sort of HAL 9000 that is both conscious, intelligent, and benevolent, it might lead you to some weird places.
But that’s on you. The rest of us can handle it. I mean, I ask Google Gemini questions like "What film in my inventory should I take to boy scout summer camp, and why? " and then evaluate what it says. Sometimes it has good points I hadn’t considered, and other time it’s just parroting the party line but dressed up in better language than the original Reddit threads it read. But ultimately I’m not taking what it says to the bank; it’s just a tool.
I wouldn’t ask it for relationship or job advice; like I said, it’s neither conscious nor intelligent. At best it’s going to read the conventional wisdom and parrot it back to you. But it may also read a bunch of half-cocked assholes on Reddit, Twitter, and blogs, and present you with a bunch of toxic nonsense.
You basically have to have a very basic understanding of how LLMs work to see WHY they’re an awful choice for that sort of thing, and why they’re amazing for say, asking it to identify the top 5 items in the Federal Budget that have to do with swim goggles or something.
My friend does not believe AI is sentient. He could probably explain transformers better than most people here. This “it couldn’t happen to me” attitude is precisely what I’m concerned about. My friend believes he is skeptical of AI. Without getting too into the weeds, though, he believes that LLMs have been fed in so much information and are so efficient at distilling it, that it’s worth taking those results seriously, even if they seem at odds with what others are saying.
For instance, a few months ago I spent quite a while talking to ChatGPT about my retirement. I’m not a financial wizard, but I was definitely in the “AI is great for educating yourself quickly on any topic” camp. It didn’t take long before it started stating things with confidence, based in part on my feedback that that’s what I wanted it to say. When fact checking it later, I discovered that it had no reason to be so confident.
The first time I played with AI, I was thrown off-guard by the conversational tone. I put in an idea for a book I was pondering writing, and AI responded as if I had the greatest idea ever conceived by man. The responses were overly enthusiastic, suggesting that it would help me with every aspect of the process, right up to winning the Peabody award. It made it all sound so easy.
My ego didn’t catch on to what it was doing at first, which was essentially making me feel good and nothing more. Thankfully, I learned more about AI and realized it was essentially blowing smoke up my ass. But I can see how someone could easily be taken in by AIs “advice” and start trusting it. I haven’t used it much since, but I’ve certainly seen people who rely on it to answer every question they have. That scares me.
I would just want to know what is actually going on with him that leads you to believe he’s dealing with AI Psychosis. What psychological problem has it made worse? Or what about his behavior has changed?
I don’t disagree with your warnings about using it for decision making. It very much is not an expert and tends to lack humility. The way these things are designed makes them more interested in sounding convincing, and expressing uncertainty gets in the way of that.
But this sort of thread doesn’t really work without actual examples.
Paranoia, conspiratorial thinking, cutting off anyone who suggests that he might not be seeing things clearly. For what it’s worth, he’s been diagnosed by a licensed psychiatrist, who he’s stopped seeing. I don’t want to get into specifics but the psychosis is real. The AI angle is open to interpretation I suppose, but people who are concerned about him are in concensus. His doctor told him to limit his AI use before he stopped going.
Plenty of people have propensities to mental health problems - making them vulnerable to particular triggers. It wouldn’t surprise me that the same people who are vulnerable to (say) cult recruitment techniques (or Amway) might be triggered into a mental health crisis by an automated system that agrees with (or even exaggerates) every notion that comes to mind. And while you might never run into a cult recruiter, AI is potentially on your phone.
Do ravens or late night talk shows have a documented history of creating or worsening psychological symptoms in significant numbers of people? A one-off does not a syndrome make.
There’s even an obvious mechanism here. Everyone talks about AI’s often sycophantic behavior. And we acknowledge that being surrounded by fans and yes men often leads celebrities to lose touch with reality. We acknowledge how enablers can make psychological problems worse.
In terms of “not using AI for decision making,” I would argue that at some level, that’s unavoidable. If your prompt is, “Write me a unit test for this function,” you’re effectively letting the AI make a bunch of small decisions about how that test is structured. Sure, you can choose whether or not to accept or reject the suggestion, but those decisions it makes are going to creep into your codebase (and therefore, your life).
Likewise, when I was using ChatGPT for meal planning (super useful, I must admit), it would curate suggestions. Yes, I was the final decider, but my decision was based on information that the chatbot controlled.
So I don’t think many people would sit down and say, “Should I divorce my husband” and listen to what it says. But they may ask, “What are some signs that my husband is cheating on me?” And then they try to fit reality to the pattern that the chatbot gives them. Is he staying out late with friends? Is he hiding his phone from me? Maybe the chatbot says “If you can give me some more information about what’s going on, I can give you more specific answers.” So you do, and it feeds into it – yes, your husband coming home late from the gym last week could be a sign that he’s cheating.
Next thing you know, you’re convinced he’s cheating. You won’t listen to his reasonable explanations, because you’ve already decided he’s a liar. You won’t listen to his friends, because of course they’re protecting him. You won’t listen to your family, because they always liked him more than you anyway. Etc, etc.
I’m sure this happened plenty before AI, but now it’s just so easy.
I don’t understand how this worldview is helpful. Let’s say that 10% of the population is predisposed to severe anxiety. Let’s also say that of that 10%, half can have anxiety attacks triggered by excessive caffeine.
Should we not warn people about excessive caffeine consumption because, hey, those people already had underlying issues anyway?
“Warn them” is fine, as long as rules and regulations aren’t created banning or significantly limiting access to caffeine for the other 95% of people. And that’s the way things tend to go in the litigious US nanny state.
This is a big problem in the patent world right now. The pro se help unit of the patent Office is being overrun right now with inventions (that aren’t inventions) that will revolutionize or completely change the world.
I’m involved in a pro-bono patent organization. We are also being inundated with the same thing to the point where it is affecting our main purpose.
I’m not going to diagnose anybody because I don’t have to expertise to do any such thing, but I’m talking with many people who certainly fit the description of AI Psychosis. They have invented something that will change everything. People will be beating down their door to invest in or begging to buy their thing.
However, an invention isn’t an idea. An invention is an idea (or concept) plus implementation. AI often does not give you that implementation part. And people (who may be undergoing AI psychosis) are not happy to hear that they are not at the point where they have an invention. From their point of view, they have a 100 pages of invention and an AI telling them there is an invention and they are the one in a position to save the world.
I’m not seeing it in paying clients yet. Hopefully, there aren’t patent attorneys out there taking advantage of paying clients undergoing AI Psychosis.
Television (and before that radio, and movies) have caused vulnerable people to believe they have a real relationship with the people they see/hear in apparently intimate circumstances. As technology improves to make the simulation of real connection more detailed, more personal and more responsive, the percentage of people vulnerable can be expected to increase. I’m not suggesting any particular regulation or restriction - just an awareness of the potential issue.
Okay. So let’s not use actual medical conditions to refer to sort of pop-science ideas about an effect we want to describe. Psychosis is a real thing. AI does not induce psychosis. Now, if you had someone who actually had psychosis, and they talked to a sycophantic chat bot, that would be a problem - validating the delusions could indeed cause greater problems. But that doesn’t mean that people who take what AIs say uncritically are undergoing “psychosis” and we shouldn’t use that term.
Secondly, the the fact that you’re aware of how AI can lead you astray makes you pretty innoculated against the effect that you’re worried about. You likely do not need to actually avoid using AI because being aware of the path you’re worried about makes it hard for you to be steered down that path. The people who are most vulnerable to AI psychofancy are people who are ignorant that AI is not omniscient (and that’s extremely hard to believe in the current culture where AI is one of the most hated things in the world and people dramatically exaggerate the dangers and underrate the uses) and people who are vulnerable to being told their worldview is correct.
The thing is - we already have view-affirming media everywhere that already has a deep hold on the latter group. I’m not sure it makes sense to invent a new concept of “AI psychosis” unless we’re willing to say there’s social media psychosis and fox news psychosis and qanon psychosis. In fact, a responsible AI may be the best thing for them. We don’t have the manpower to cult deprogram 50 million people from living a life immersed in fox news and qanon and social media. But Claude could honestly do some real good there.
And before you accuse me of naming a specific chatbot as though it’s irrelevant because all AI is the same and it’s like picking what color cars you’re condemning, Anthropic genuinely has a different philosopy than the rest of the frontier AI companies and builds FAR more responsible AI. Claude is basically engineered from the ground up with avoiding what you’d call AI psychosis. People are grouping LLMs and “AI” into one bucket without distinguishing between different company and different systems. Google/gemini is pure sycophancy. To the point where it’s sickening and irresponsible. Claude, on the other hand, is more epistemically responsible than any person that almost everyone interacts with in their life.
AI is a useful tool. Particular implementations can be dramatically more or less responsible and more or less sycophantic. The real argument we should be making here is that AI should be designed to be less sycophantic and people should use critical thinking skills when using it. I know that’s a big ask, but then so is telling people not to use AI indefinitely