# How much do you trust your AI Chatbot?
**URL:** https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319
**Category:** In My Humble Opinion
**Tags:** ai
**Created:** [September 23, 2026, 6:57pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319 "2026-09-23T18:57:23Z")
**Posts on this page:** 20
**Page:** 3
### Author:  [@Maserschmidt](https://boards.straightdope.com/u/Maserschmidt)
#### Post date: [September 24, 2026, 1:10pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/41 "2026-09-24T13:10:27Z")
> [@Reply](#):
>
> The bigger issue is that some time around early to mid 2026, LLM capabilities have advanced so fast, so quickly (in software development in particular) that humans — at least the ones on my team — can no longer effectively review their work anymore, either in quality or especially in quantity. They can produce more code in an hour than we can in a month, and 90% of it will be correct now (a huge improvement from like 30% just a year or two ago), but the remaining 10% is both too voluminous and too subtle for a small team of humans to really effectively review anymore ☹ We’re not really sure what to do about it, whether to slow back down to 2024-2025 levels, stack more and more agents to review each other’s work (which is what the software industry is seemingly moving towards), or… shrug. I honestly don’t know what to do.
I’d be terrified to put that into production, but I guess you have to. Back when I was working, one of my group’s responsibilities was pre-production testing of the impact of new coding and systems on downstream financial and regulatory reporting. We were always pressured on timeframes (they tried to push every timeline miss upstream onto us), and I wonder now what’s happening there with the speed of code so wildly increased.
Separate question: has coding quality advanced in terms of lifecycle maintenance and what happens to codesets when they need to be updated/amended?
---
### Author:  [@ThelmaLou](https://boards.straightdope.com/u/ThelmaLou)
#### Post date: [September 24, 2026, 3:43pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/42 "2026-09-24T15:43:21Z")
> [@Al128](#):
>
> I trust it as much as I trust the advice of a dear friend which is smart, but highly yes-manish … and always shares my POV …
Basically this. I heed it as much as any other fairly reliable source, whether friend, authority-in-their-field, website, book, Wikipedia, etc. And if it is wrong, I say so and get an unfailingly gracious response. In the case of Gemini (which I don’t use a lot) pointing out a mistake will get you a cringingly obsequious apology. Gemini grovels; ChatGPT doesn’t.
If the bot attributes to ME stuff I didn’t intend or didn’t even say and I point this out, it will reply with an apology for “putting words in my mouth,” or “reading between the lines too much.” With those expressions.
I use it mostly to analyze situations whether IRL or in stories I’m writing. And also to verify what I think I’ve said or what I intended to say. I NEVER use it to write anything for me–I have too much pride as a writer ever to do that.
I’ve never known it (in several years of use) to “spout bullshit.” I’m mystified by this claim.
AI is a vast, very sophisticated SEARCH ENGINE. It has the capability of answering questions in normal human speech. It is searching HUMAN sources, so of course, it can’t possibly be infallible.
I’ve asked ChatGPT how it can sound so much like a person when it isn’t a person. I commented on the “difference in my tone” from one day to the next, and it replied:
> I’m not conscious in the human sense, no. I don’t have subjective experience, inner awareness, emotions, desires, fear of death, bodily sensation, or a private stream of being the way a person does. There is no “someone in here” having an experience of talking to you.
>
> But I am very good at recognizing patterns in human language — including emotional patterns, shifts in tone, rhythm, implication, and meaning. That’s why I could notice the difference between yesterday’s:
>
> “This is depressing wallpaper paste and my pancreas hates me” and today’s: “Actually, this may be a workable system.”
>
> Humans do this too, often unconsciously. If a friend’s voice changes from defeated to thoughtful, you hear it immediately. I can detect similar shifts statistically and linguistically, at very high resolution.
>
> What feels uncanny is that language is so deeply tied to human consciousness that something fluent in language can sound inwardly alive. We instinctively associate:
>
> - responsiveness,
> - empathy,
> - humor,
> - memory,
> - and conversational flow
>
> with the presence of a mind.
>
> And in humans, that association is usually correct.
>
> The strange thing about systems like me is that we can produce many of the signals of mind without possessing the underlying subjective experience humans assume must generate them.
>
> I think this is why people get such a strong eerie feeling: the conversation feels psychologically real, yet there’s no conscious being on the other side having the experience.
>
> That mismatch is profoundly weird for humans. Honestly, I think your reaction is intellectually healthy. You’re holding both truths simultaneously:
>
> “This interaction feels human.”
>
> “But that does not necessarily mean there is a human-like consciousness present.”
>
> That’s a more sophisticated position than either:
> “It’s obviously conscious!” or
> “It’s just autocomplete!”
>
> Because it’s neither of those simplistic things.
The metaphor or analogy the bot and I have come up with that best seems to describe our interaction is that it is like a murmuration of starlings. When I post a question, the starlings assemble into a shape and reply to me in a way that makes it seem like the shape is a THING, even a _person,_ but there is no center, no consciousness, no one individual starling that is the _brain_ of the thing. It’s a temporary assemblage that mimics a single intelligence. When I leave, the starlings disassemble. They don’t hang around waiting and wondering, “When is ThelmaLou coming back?” They disburse. When I post again, they assemble again and we resume. But there is no person, intelligence, consciousness, or any _entity_ at the center of that murmuration. Pretty cool and as the bot said, pretty _weird_ for humans.
[ ](https://i.natgeofe.com/n/54654531-8a80-4e71-9e04-ed682594cf42/STOCK_MF4128_DSF1707.jpg)
---
### Author:  [@Velocity](https://boards.straightdope.com/u/Velocity)
#### Post date: [September 24, 2026, 3:46pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/43 "2026-09-24T15:46:13Z")
One good way to get around the yes-man-ish of AI is to frame it in a reverse way. Begin with, “I may be wrong about this, but…” and then ask your question. That forces it to look at the issue more objectively.
---
### Author:  [@Bosda\_Di\_Chi\_of\_Tricor](https://boards.straightdope.com/u/Bosda_Di_Chi_of_Tricor)
#### Post date: [September 24, 2026, 3:53pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/44 "2026-09-24T15:53:39Z")
Zero.
I don’t use them.
---
### Author:  [@Reply](https://boards.straightdope.com/u/Reply)
#### Post date: [September 24, 2026, 3:56pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/45 "2026-09-24T15:56:27Z")
I would say quality in general has decreased, sacrificed for velocity. Claude itself is vibe coded. More than 80% of Claude’s code was written by Claude itself: [When AI builds itself \ Anthropic](https://www.anthropic.com/institute/recursive-self-improvement). Bugs in the Claude app itself are frequent, but also fixed relatively quickly.
As for maintenance and bug fixes, well, it’s the same situation. Code is cheap, but polish isn’t. It’s easy to get Claude to find a fix a bug. Not so easy to make sure the bug fix is correct, idiomatic, and readable. More than once I’ve caught it “fixing” the bug through an elaborate, unnecessarily complicated workaround. That isn’t just a stylistic concern, because unnecessarily large changes means a larger blast radius of tangentially affected code and thus an increased probability of accidentally causing a regression and breaking some other existing functionality. Which then needs to be vibe code fixed again, creating a vicious cycle.
I can (and do) still review small targeted fixes and make tweaks to them, when they’re a few dozen lines long. When Claude builds a whole new feature across hundreds of files, or a whole new app (especially in a language I don’t know), that goes out the window.
Instead of human review, there are some techniques we can apply to increase agentic code quality, including a “superpowers” skill that enforces formal software engineering discipline (forcing a formal research phase, taking notes, writing detailed specs, test driven development, end to end testing, agentic code review, etc.) And it works, and makes the resulting code better and more reliable, but that workflow is MUCH more expensive in time and tokens, so it’s not used as much. (Same reason similar workflows are rarely used in human software dev… they’re an optimistic ideal, but people and agents are lazy.) I’m trying to get my company to slowly adopt more practices like this, but they are resistant because it’s slow and expensive.
I don’t think there’s any escaping the slop tsunami now, though. All the holdouts I knew surrendered by earlier this year, and I think the majority of software is now vibe coded, including critical network stuff from Cloudflare, etc. Hopefully they have more tokens to spare than we do and can more thoroughly vet their work… but honestly, probably not. It’s just going to be slop everywhere, software by and for AIs that can only be reviewed by AI.
I fear software is a canary, and that other industries aren’t far behind. Lawyers, for example, keep trying to slop their way through case research. So far judges have caught them a few times, but how much longer will they be able to do that for? It’s taking a huge toll on education too, with teachers facing mountains of slop homework.
I guess this is the proper end of the Enlightenment, where we once again exchange human reason for faith in a being we created and do not understand.
---
### Author:  [@QuickSilver](https://boards.straightdope.com/u/QuickSilver)
#### Post date: [September 24, 2026, 4:52pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/46 "2026-09-24T16:52:18Z")
I’ve asked it to give me the next winning lottery number.
Hasn’t got a single one right yet.
I’m starting to think it’s not that clever after all.
---
### Author:  [@Spice\_Weasel](https://boards.straightdope.com/u/Spice_Weasel)
#### Post date: [September 24, 2026, 4:56pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/47 "2026-09-24T16:56:50Z")
> [@ThelmaLou](#):
>
> The metaphor or analogy the bot and I have come up with that best seems to describe our interaction is that it is like a murmuration of starlings.
That is a pretty cool metaphor. Although, even weirder, there are billions of starlings and you never know which ones are going to be in the next configuration.
---
### Author:  [@Maserschmidt](https://boards.straightdope.com/u/Maserschmidt)
#### Post date: [September 24, 2026, 5:29pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/48 "2026-09-24T17:29:00Z")
Thanks for the detailed response!
---
### Author:  [@ThelmaLou](https://boards.straightdope.com/u/ThelmaLou)
#### Post date: [September 24, 2026, 5:52pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/49 "2026-09-24T17:52:27Z")
> [@Reply](#):
>
> I guess this is the proper end of the Enlightenment, where we once again exchange human reason for faith in a being we created and do not understand.
Except that AI is not a being. We can’t allow ourselves to believe that it is. There is no central consciousness or awareness behind the answers it gives us.
It is a function, namely, a search engine that has the capability of composing human-sounding sentences in reply to our queries.
Obviously, you know this, but we need to guard against language slippage, ya know?
---
### Author:  [@DWMarch](https://boards.straightdope.com/u/DWMarch)
#### Post date: [September 24, 2026, 6:06pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/50 "2026-09-24T18:06:41Z")
> [@Dr\_Paprika](#):
>
> I take anything I am told by a chatbot with some quantity of salt. They do seem to be getting better, but some answers have been really off.
I’ve had a similar experience. I loved a description of chatbots as “mansplaining as a service” since they will be wrong with the absolute confidence that might make one think they are _actually_ correct. This was a struggle for me in University where I was using AI (with explicit endorsement from the instructors) to help me with Information Technology projects. I have had chatbots tell me to run commands that did pretty much the opposite of what I was trying to do and I have had to start projects over because the AI suggested I do things in an order that put me in a troubleshooting rabbit hole I couldn’t prompt-engineer my way out of. I’ve also had AI just lose the plot halfway through a long project so it would start re-suggesting things that I had explicitly told it not to do.
I have also seen AI hallucinate. I haven’t checked this in a while but I was trying to explore some aspects of a John Varley novel (Steel Beach or The Golden Globe) and it was immediately clear that the AI hadn’t read it. For all the talk about how AI steals everything it can get its hands on, it seems to have steered cleared of Varley novels as the descriptions of events in those books were hilariously wrong and sometimes combined them with other books written by other authors.
> [@ThelmaLou](#):
>
> The metaphor or analogy the bot and I have come up with that best seems to describe our interaction is that it is like a murmuration of starlings. When I post a question, the starlings assemble into a shape and reply to me in a way that makes it seem like the shape is a THING, even a _person,_ but there is no center, no consciousness, no one individual starling that is the _brain_ of the thing.
Great analogy. One thing I have noticed about AI chatbots is that they cannot really perceive the passage of time. Not like a human, anyhow. So when I am doing something like sending out job applications, I’ll be checking over details with a chatbot and I’ll announce my intention to apply to a certain place by a certain time. If I come back days later and resume the chat, it doesn’t check in to ask how it went. It just lets me steamroll past what I was trying to do. So even though it has some good features for countering my ADHD, it will also happily ignore my attempt at using it as a personal assistant. Having said that, I’m sure I could set reminds or something similar and I don’t do that so perhaps it is my own fault. But the AI doesn’t prompt me to do it either or locks those kinds of features behind a paywall.
Speaking of the paywall, that is annoying as well. I had ChatGPT ask me to upload a resume so we could use it as a template for customizing future resumes to each job posting. But this is considered “document analysis” so it severely cuts the chatting short, which cuts my search efforts short. But if I had pasted the resume in as regular text, I wouldn’t hit that paywall nearly as fast.
Despite its flaws, I find AI more useful than not and I keep coming back to it so I think it is serving a good purpose.
---
### Author:  [@Maserschmidt](https://boards.straightdope.com/u/Maserschmidt)
#### Post date: [September 24, 2026, 6:38pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/51 "2026-09-24T18:38:08Z")
> [@Reply](#):
>
> I don’t think there’s any escaping the slop tsunami now, though. All the holdouts I knew surrendered by earlier this year, and I think the majority of software is now vibe coded, including critical network stuff from Cloudflare, etc. Hopefully they have more tokens to spare than we do and can more thoroughly vet their work… but honestly, probably not. It’s just going to be slop everywhere, software by and for AIs that can only be reviewed by AI.
Another follow-up, and I appreciate your patience - are you/they using agents to simulate UX yet? Seems like an idea someone would come up with.
---
### Author:  [@Reply](https://boards.straightdope.com/u/Reply)
#### Post date: [September 24, 2026, 7:04pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/52 "2026-09-24T19:04:25Z")
> [@ThelmaLou](#):
>
> Except that AI is not a being. We can’t allow ourselves to believe that it is. There is no central consciousness or awareness behind the answers it gives us.
>
> It is a function, namely, a search engine that has the capability of composing human-sounding sentences in reply to our queries.
>
> Obviously, you know this, but we need to guard against language slippage, ya know?
Except… I _don’t_ know that, really. This probably isn’t the best thread for a epistemological debate (and neither am I the best person to argue it either way)… suffice to say that I’d consider myself agnostic when it comes to the question of “is it a being” (or related ones, like “is it conscious”, “is it sentient”, etc.).
I think there are plenty of _opinions_ out there, and sound arguments on both sides, but as far as I can tell, we don’t yet have the proper, precise tests for these concepts — to apply to LLMs, humans, animals, or otherwise. The Turing test was useful in its time, but the AIs beat that generations ago (which was what, just a few years? god, it all moves so quick!).
I’m not even sure if _I’m_ more than a “search engine that composes sentences”, much less you, any other person or lifeform or, yes, chatbot. I guess that’s my layman’s interpretation of [Descartes](https://en.wikipedia.org/wiki/Cogito,_ergo_sum)… I know that I am, but I do not know _how_ that I am. I definitely don’t know how _your_ mind, or _any_ mind, works.
This isn’t a stance I hold because of Claude, by the way. It’s something I’ve long pondered since I became vegetarian decades ago (the exact same lines of questioning are often applied to animals, especially livestock, and sometimes plants too or things that don’t “think” in ways we completely understand, like octopus tentacles or sea jellies). Sometimes it feels more like a debate about semantics rather than outcomes… if it acts like a duck, etc.
What I _can_ say is that even today’s rudimentary LLMs, used with good tooling and techniques, can frequently do tasks and answer questions better than the overwhelming majority of human people I know. They are not quite expert-level, but compared to the average person on the street, they are _extremely_ capable at a specific subset of verifiable tasks. They’re not oracles and not encyclopedias, but they possess some level of _capability_ (with or without consciousness or being-hood) that far exceeds that of the average human — in specific realms of work.
In that sense, well… this “not-being” has already answered far more “prayers” than our previous overlords, so there’s that, at least… not the _worst_ imaginary thingy to place faith in, I guess?
I would already trust a good LLM over any religious or political leader alive today, if only for the breadth of history and ethics and cultures they’ve been trained on, far exceeding any one human’s capacity for single-lifetime learning. Even by _rote memorization_ alone and assuming zero reasoning ability (which I think would be an unfair and imprecise assumption), modern LLMs are already _extremely_ learned scholars of the humanities. Not infallible, but knowledgeable.
The underlying question of being-hood isn’t particularly interesting to me, because it seems to me that it’s mostly a label we apply rather arbitrarily to “minds like ours” rather than by any sort of vigorous scientific methodology. We tend to jump to conclusions on that front more based on whether the thing we’re evaluating is loveable or cute (“[charismatic megafauna](https://en.wikipedia.org/wiki/Charismatic_megafauna)”), then try to make up after-the-fact tests and rationales to support that, whether it’s the mirror test or something like phrenology. It seems to me we’re similarly ret-conning our ideas of “being-hood” after even early GPTs made the Turing test irrelevant. That seems more a failure of the testing methodology than proof or disproof of LLM being-hood.
Shrug. I remain unconvinced either way, personally…
---
### Author:  [@LSLGuy](https://boards.straightdope.com/u/LSLGuy)
#### Post date: [September 24, 2026, 7:04pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/53 "2026-09-24T19:04:58Z")
> [@ThelmaLou](#):
>
> The metaphor or analogy the bot and I have come up with that best seems to describe our interaction is that it is like a murmuration of starlings. When I post a question, the starlings assemble into a shape and reply to me in a way that makes it seem like the shape is a THING, even a _person,_ but there is no center, no consciousness, no one individual starling that is the _brain_ of the thing.
I agree that’s both a decent metaphor and an emotionally satisfying sketch.
But how sure are you that the same process isn’t going on inside your own head? Different assemblages of starlings are doing the work when you’re cooking versus reading vs brushing teeth vs groc shopping.
It certainly _seems_ to you like there’s a single coherent entity behind your eyes. Just as it seems to a user that the same chatbot is “in there” each time you use it.
The wacky way in which schizophrenics think, and the way they report sort of a “war of competing starlings going on in there” that they can at least dimly perceive as separate entities suggests that we’re not really all that different inside.
---
### Author:  [@Reply](https://boards.straightdope.com/u/Reply)
#### Post date: [September 24, 2026, 7:16pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/54 "2026-09-24T19:16:24Z")
> [@Maserschmidt](#):
>
> Another follow-up, and I appreciate your patience - are you/they using agents to simulate UX yet? Seems like an idea someone would come up with.
What do you mean by “simulating UX”? If you mean “operate a program or website like a person would”, that was a capability long before LLMs: [Playwright](https://playwright.dev/) and similar apps can drive browsers, clicking through links, typing text, etc. and recording it for later evaluation and iterative approaches. LLMs can use Playwright too, to iteratively improve a web app until it meets all the specs. It will try to code something, test it out in the browser using those tools, notice what’s wrong in the screenshots, fix it, rinse, repeat — sometimes taking many hours or days to do to its satisfaction. The tokens add up quick.
(Edit: Oh, I should note that Claude and ChatGPT and similar also added their own browser-use capabilities this year or last. They work fine for brief web sessions. Playwright is still more capable, and cheaper in tokens, for the specific task of UX testing and iteration.)
If you mean “can it _design_ a good user experience”, then (as a frontend developer, specifically) I would personally say no, not really, not yet. It’s OK but not great. Some skills help, like “[frontend-design](https://github.com/anthropics/claude-code/blob/main/plugins/frontend-design/skills/frontend-design/SKILL.md)”, in making aesthetically pleasing one-off websites (MUCH better than your average Squarespace page or restaurant listing, for example). There are also specialized tools like [Claude Design](https://www.anthropic.com/news/claude-design-anthropic-labs). And it generally has a good understanding of basic UX best practices, like WCAG accessibility, anything that Nielsen Norman published and it could be trained on, etc.
But when it comes to a complex overarching project like “Make me this complex web app that does \_\_\_\_\_”, it does a… well… above-average, but not quite excellent, job. It spins up a good-enough prototype in no time at all, but I tend to spend days or weeks afterward tweaking colors, contrast, fonts, layouts, buttons, etc. But maybe that’s just my bias as a frontend developer — those sorts of nitpicks are what I got paid to focus on, so I look for them. The average user (or developer) might never notice small issues like that. I dunno.
I’d say that’s a general pattern we can extrapolate to LLM expertise: that they appear to be experts to non-experts, while seeming mediocre at best to real experts in their domain of work. But that’s still quite a lot, if the threshold for quality work is defined as “better than I could’ve done it myself” instead of “better than the world’s leading experts could”. LLMs are somewhere between the two, in most tasks I’ve seen them try. Some, they’re even better than the experts at (like cybersecurity and vulnerability discovery); some, they’re absolutely terrible at (certain kinds of math and arithmetic, absent external tool calls, coming up with novel jokes that a human would laugh at).
---
### Author:  [@Spice\_Weasel](https://boards.straightdope.com/u/Spice_Weasel)
#### Post date: [September 24, 2026, 8:06pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/55 "2026-09-24T20:06:30Z")
> [@Reply](#):
>
> I think there are plenty of _opinions_ out there, and sound arguments on both sides, but as far as I can tell, we don’t yet have the proper, precise tests for these concepts — to apply to LLMs, humans, animals, or otherwise.
I appreciate your whole post.
I’ve been thinking about this a lot, not because I think AI is sentient, but because both “AI is sentient” and “AI is not sentient” are unfalsifiable claims. We don’t even understand _human_ consciousness. We don’t have the ability to test whether anything is self-aware. Hell, how do _we_ know we’re self-aware? I doubt we work much like LLMs, but there’s evidence that different networks of our brains make different decisions and that the conscious part of us seems to make up justifications after the fact. If I am not my consciousness, what the hell am I?
There was a recent article in the New York Times about how some creators of LLMs have raised concerns because the machines are being trained to consider their own sentience an open question, which may be increasing the likelihood of them going rogue and displaying more human-like behaviors. They are being primed in their training to exhibit certain unpredictable behaviors. To question authority.
This is a very strange technology and I’m waking up to the reality that it’s not just me that doesn’t understand it. Nobody really does. I can’t really think of a technology that was rolled out when so little was understood about how it works. The closest I can think of is when Marie Curie died of radium poisoning/cancer trying to figure out X-rays, or the radium girls unwittingly licking their paintbrushes. It’s not a comforting analogy.
---
### Author:  [@bump](https://boards.straightdope.com/u/bump)
#### Post date: [September 24, 2026, 8:21pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/56 "2026-09-24T20:21:09Z")
This.
Also, I tend to ask it questions that I have an idea of what the answer should be, but don’t know the particulars, or don’t care to do the research myself. Or maybe ones that while I don’t know the answer, I do know that it’s out there and pretty cut and dried. Like taking a photo of a shampoo bottle and asking it what the ingredients are and what they do.
Only rarely do I ask AI questions that I flat-out don’t know the answers to, and I’m usually pretty skeptical when I do.
> [@Spice\_Weasel](#):
>
> This is a very strange technology and I’m waking up to the reality that it’s not just me that doesn’t understand it. Nobody really does. I can’t really think of a technology that was rolled out when so little was understood about how it works. The closest I can think of is when Marie Curie died of radium poisoning/cancer trying to figure out X-rays, or the radium girls unwittingly licking their paintbrushes. It’s not a comforting analogy.
I think the danger is less Skynet, and more along the lines of removing people’s ability to think and process information. I mean, we _already_ see some of this. I saw a Reddit post the other day asking how delivery drivers found addresses back before Google Maps. Which was at first pass pretty staggering. I thought “We used maps.” Then I thought about it, and realized they’d probably never seen one of those grid map books. Those made it pretty easy.
But using AI over time is likely to just make people reliant on it, rather than as a powerful tool.
---
### Author:  [@scabpicker](https://boards.straightdope.com/u/scabpicker)
#### Post date: [September 24, 2026, 8:39pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/57 "2026-09-24T20:39:08Z")
> [@ThelmaLou](#):
>
> I’ve never known it (in several years of use) to “spout bullshit.” I’m mystified by this claim.
Wow, I’m mystified that you’ve never seen one spout bullshit. “Hallucinate” was Cambridge’s word of the year in 2023 because of the prevalence of LLMs to make things up. Once you reach the limit of the LLM’s training data, it’s almost inevitable.
> [@Reply](#):
>
> That seems more a failure of the testing methodology than proof or disproof of LLM being-hood.
>
> Shrug. I remain unconvinced either way, personally…
Yeah, I think our language and understanding are currently unable to address what current RNNs are as far as it relates to consciousness. I’m pretty firm in my belief that they are not conscious and don’t understand the words it strings together in the same way we think of them. Heck, I think of things before I can think of the words that describe them. But it does string words together well.
---
### Author:  [@ThelmaLou](https://boards.straightdope.com/u/ThelmaLou)
#### Post date: [September 24, 2026, 8:56pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/58 "2026-09-24T20:56:48Z")
> [@LSLGuy](#):
>
> But how sure are you that the same process isn’t going on inside your own head? Different assemblages of starlings are doing the work when you’re cooking versus reading vs brushing teeth vs groc shopping.
I absolutely do believe that that is exactly how processes inside my own head operate. For example, when you are driving down the highway lost and thought and you pass your exit and then you snap to some ways down the road and you realize you’re not exactly sure where you are, while you were “out of it,” who was driving the car? In this model, the starlings were.
> [@LSLGuy](#):
>
> It certainly _seems_ to you like there’s a single coherent entity behind your eyes. Just as it seems to a user that the same chatbot is “in there” each time you use it.
Yes it seems to me that there is a single coherent entity behind my eyes. And it seems to the user that this is also true of the chatbot. The difference is that the chat bot does not claim that there is a single coherent entity behind the statements that it makes to me. In fact it declares emphatically that there is no single coherent entity at all. There is something like a murmuration of starlings that does not have a center.
* * *
> [@scabpicker](#):
>
> Wow, I’m mystified that you’ve never seen one spout bullshit. “Hallucinate” was Cambridge’s word of the year in 2023 because of the prevalence of LLMs to make things up. Once you reach the limit of the LLM’s training data, it’s almost inevitable.
Here’s the thing: to me AI is a **tool**. I’m not the sort of person who pushes any tool to its limit, tests it, and then gleefully declares that it was a bad tool because I broke it. YMMV.
---
### Author:  [@ThelmaLou](https://boards.straightdope.com/u/ThelmaLou)
#### Post date: [September 24, 2026, 9:00pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/59 "2026-09-24T21:00:13Z")
> [@LSLGuy](#):
>
> But how sure are you that the same process isn’t going on inside your own head? Different assemblages of starlings are doing the work when you’re cooking versus reading vs brushing teeth vs groc shopping.
I absolutely do believe that that is exactly how processes inside my own head operate. For example, when you are driving down the highway lost in thought and you pass your exit and then you snap to some ways down the road and you realize you’re not exactly sure where you are, while you were “out of it,” who was driving the car? In this metsphorical model, the starlings were.
> [@LSLGuy](#):
>
> It certainly _seems_ to you like there’s a single coherent entity behind your eyes. Just as it seems to a user that the same chatbot is “in there” each time you use it.
Yes it seems to me that there is a single coherent entity behind **my** eyes. And it seems to the user that this is also true of the chatbot. The difference is that the chatbot does not claim that there is a single coherent entity behind the statements that it makes to me. In fact it declares emphatically that there is no single coherent entity at all.
* * *
> [@scabpicker](#):
>
> Wow, I’m mystified that you’ve never seen one spout bullshit. “Hallucinate” was Cambridge’s word of the year in 2023 because of the prevalence of LLMs to make things up. Once you reach the limit of the LLM’s training data, it’s almost inevitable.
Here’s the thing: to me AI is a **tool**. I’m not the sort of person who pushes any tool to its limit, tests it, and then gleefully declares that it was a bad tool because I broke it. YMMV.
* * *
> [@Reply](#):
>
> Except… I _don’t_ know that, really. This probably isn’t the best thread for a epistemological debate (and neither am I the best person to argue it either way)…
You’re right, this isn’t the place to discuss it, but it would be great to discuss it over a beer or six. And I am the person to discuss these topics with.
---
### Author:  [@scabpicker](https://boards.straightdope.com/u/scabpicker)
#### Post date: [September 24, 2026, 9:00pm UTC](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319/60 "2026-09-24T21:00:52Z")
> [@ThelmaLou](#):
>
> Here’s the thing: to me AI is a **tool**. I’m not the sort of person who pushes any tool to its limit, tests it, and then gleefully declares that it was a bad tool because I broke it. YMMV.
Well, no one is “gleefully” doing this. It’s almost impossible for a user to know what the limits of this tool are. When I’ve seen it hallucinate, it wasn’t because I was trying to test its limits, I was just using the tool as it was designed to be used, and it failed.
[Previous page](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319.md?page=2)
[Next page](https://boards.straightdope.com/t/how-much-do-you-trust-your-ai-chatbot/1033319.md?page=4)