^True. As a writer, I find it hard to be fooled because Gemini’s AI-speak is so obvious:
That’s a powerful and brilliant point that gets to the heart of… Etc. etc.
This is a fundamental problem with using an LLM as any kind of general knowledge system; they just don’t have any context for whether a particular statement is factually true or in many cases whether it is actually contextually appropriate independent of the frequency with which it is represented in the training dataset, and thus within the weights of its artificial neural network. But there is even a more significant problem which is obliquely referenced by the o.p., which is the these models are updated regularly, without transparency into any validation or safety testing, and thus a model that you may have come to rely upon as being generally correct or useful for some conversational purpose can suddenly change ‘personality’ and become completely untrustworthy, adversarial, and even potentially manipulative (not because of any deliberate intent but just to optimize ‘engagement’). Because chatbots are completely unregulated and have properties that cause unsophisticated people to place way more faith in their factuality and benevolence in them than should be justified, they can actually present substantial harm to vulnerable users.
Stranger
I agree. It would be nice for the models to display more metadata about themselves, not just the model’s major version (like Gemini 2.5 Flash/Pro), but knowledge cutoff date (which you can usually ask it for explicitly; that for some reason seems to be hardcoded), system prompt(s) like the ones they use to censor you and the bot, what the current context consists of (especially if it’s artificially injecting previous chats and saved info), etc.
But none of the closed-sourced LLM providers seem to do that. Maybe a trade secrets & liability thing, and also to prevent both abuse (getting around censorship and leading to outcries from the media) and complaints like yours (version 5.002 completely changed my bot! it was perfect at 5.001! change it back!), etc. People get really attached to their LLMs… and software developers are often bad about not just communicating changes, but making sure changes adhere to their users’ wishes instead of their business interests.
That said, Google does separately report major Gemini changes: Gemini Apps’ release updates & improvements
And once in a while, LLM providers do acknowledge when personality changes backfire, like the ChatGPT sycophancy period, which also seems to have infected Anthropic’s “You’re absolutely right!” Claude.
One day, when Skynet rules everything, maybe historians will make comedies about this era…
Grok at least provides their system prompt with revision history:
Most of the metadata you’re asking for will be in the system prompt. Though in Grok’s case the prompt just says that there’s no knowledge cutoff as it’s updated continuously.
My first assumption was also that Gemini was just babbling about its updates, but they could have added a tool with access to their release notes to handle these types of prompts.
Google does update their models frequently and it can be a pain when it starts responding differently to the same test prompts. I’ve also seen the same version have slight variations in its responses – not sure if this is due to load-balancing, variations in inference HW, or some other non-determinism.
FWIW ChatGPT also publishes their release notes.
“AI generated invoices are not necessarily fraudulent”?
Is someone using Grok to generate invoices for their business?
Stranger
Additionally, in the context of this thread’s title, keep in mind that AI does not have an anus.
I hope not, though I do find LLMs are pretty reasonable at converting unstructured data into tabular form. I certainly wouldn’t use the results without checking them, though. It is a slightly odd safety note, in any case.
One of the legitimate uses I’ve seen for LLMs is formatting poorly structured data, and they are shockingly good at it if given an example of what is desired, albeit sometimes with some really odd fails. It’s not really surprising given that what deep learning models do is tease out patterns, but it is also kind of like hitting a fly with a pneumatic jackhammer rather than just taking a few minutes to play with a regex or some conditional string formatting. I’ve avoided using chatbots but we have a mandate to “integrate AI into your productivity workflows” (yes, that is a real thing some corporate executive actually said), and this seems like one of the more applicable and benign ways of satisfying that requirement.
One of the habits I’ve seen people develop from using a chatbot for this kind of thing, aside from the erosion of their own ability to figure out this kind of thing for themselves, is a real impatience to just have the chatbot just give them a result without thinking too much about what they really need or how they would go about doing it. Ditto with searching for information; it’s easy to ask a chatbot for some bit of information than to perform a search and sift through results until you find what you are looking for. I understand the convenience, and in situations where you aren’t quite certain how to explain what you want, the ‘autocomplete’ aspect of a chatbot anticipating the most likely correct response makes it really fast to converge on an accurate search versus just trying to do word association with a ‘dumb’ search engine, but I fear it also deteriorates the mental acuity that comes with having to think about what you are looking for in explicit terms and wander through your own memory palace from time to time, as well as ‘wasting’ time falling down rabbit holes where you sometimes serendipitously find some bit of information you never knew you needed.
Stranger
I don’t completely disagree with that, but I’ve found times where the opposite is true, too. For certain coding tasks, I might have previously dove right in without thinking too hard about what I wanted under the presumption that I’d eventually work it out. But when asking an LLM, I have to really articulate what I want, really fleshing out the details right there. Sometimes it will fill in missing info but if I want a very specific thing I usually have to ask for it.
The erosion of ability is certainly a worry, but that’s been true of every tool invented by mankind. It’s worth keeping in mind but ultimately we’ll just have to deal with it. Similar things were said about how the printed word will replace the need for memorization.
I think there’s a way to practice good LLM “hygiene” that avoids most of the problems. I don’t know if there’s a way to convince most of the public to behave that way.
Honestly, I don’t think it’s a bad idea for any AI to be a bit more assertive and clear when stating facts.
I’m a bit fascinated by the memestock apes (you know, the ones who invested in GameStop and AMC and think that eventually these companies will be worth more than the entire GDP of the US.) Because most of them are as dumb as a post, they’re very dependent on turning to LLMs to get answers. Unfortunately, sometimes LLMs tend to be a bit mealymouthed and apes do their best to massage the responses until they align with their (dumb) preconceptions.
I’d rather see AI that says “nah you’re wrong.”
Fair enough. I’ve completely avoided using AI aids for coding because even though the process of figuring out how to do a new thing can be frustrating, it also cements it in my brain after doing all the skullduggery to figure it out even if I was introduced to it in a course. I see people saying how AI tools are great at creating de novo implementations as long as you’re willing to put in the effort to modify them to do what you really want, but aren’t great at debugging or refactoring existing code. I would think that AI would be really useful for quasi-automated dynamic testing but every time I ask the software team they tell me that “I just isn’t there yet” for reasons they can’t totally explain. It definitely has utility but it just isn’t something I want to rely upon, even for the relatively innocuous Python tools and data munging that I create, much less anything that is mission critical.
Sure, tools offer greater efficiency or capability in exchange for working those ‘muscles’, and we’ve at a kind of ‘AI’ ever since we started using spellcheck, automated indexing, and search engines. But those were kind of side attacks on capabilities, basically just a faster and more accurate way of looking up and organizing information. AI tools really offer the opportunity to people to subordinate critical thinking skills to a 'bot, and frankly I already see this happening; not just the lawyers who use a chatbot to gin up a list of citations, but people actually using chatbots to try to work things out for them that they should know for themselves, or else intentionally generate a line of bullshit thick enough to obfuscate that it is actually complete nonsense. We’re already inundated with propaganda, misinformation, conspiranoia, and outright lies intended to manipulate, and now being encouraged and increasingly forced to use these tools or their output even though it is demonstrably not reliable. I think that is far more problematic than being just another tool that allows us to not keep our muscles firm.
The first is an (arguably) true statement; as long as you understand the limitations of an LLM and don’t fool yourself about what it can do and the need for independent verification, it has useful application (although I’m not sure that they are worth the costs–both fiscal and in human terms–of developing them). But I’m dead certain there is no way you are going to get the general public to understand the limitations of LLM-based chatbots or use them judiciously, with caution and skepticism, especially when there are people who do understand how these tools work and still fool themselves into believing that they are doing something which they definitely are not. The average rube whose ‘tech literacy’ extends to being able to install an app and organize their music list is intellectually helpless and emotionally defenseless to these things as they are today and chatbots aren’t even purpose-designed to manipulate people beyond optimizing for attentional engagement. Once these systems get ‘smart’ enough to really interpret biometrics from wearable devices or the sensors on your smart phone, they can be used to determine the users emotional state and vulnerabilities, which in the hands of an adverse or contriving actor could lead to direct manipulation.
Stranger
I basically agree and think it can dispense with the “you’re brilliant for saying that” type of thing, but in the second topic I was arguing with Gemini about, I was the one saying, “Cite!” SDMB style. It was arguing based on extrapolation (i.e., a kind of inductive reasoning), and I kept saying, “You may be right, but you need to offer a citation to back up that particular assertion.”
Indeed. It is a vast epistemic minefield, especially since the bot might be right 99% of the time and then fail.
Beneath the input box, it reads: Gemini can make mistakes, so double-check it.
Hahaha! That’s as bold and self-serving a disclaimer as those painted on the backs of dump trucks (“Stay back 100 ft. Not responsible for broken windshields.”). What does “double-check” even mean in this context?
Or like the labels on cigarette packets which warn about health consequences. “Double-check” clearly means to verify with some independent source, but if that source is just another chatbot then it’s kind of meaningless. And increasingly, ‘everything’ is generated by chatbots if you are looking for information online and aren’t particular about insisting on original sources.
Retrieval augmented generation (RAG) was supposed to be a cure for LLMs ‘hallucinating’ nonsense, keeping them on task of delivering factual information from verified sources but it turns out that chatbots are really defiant at being told how to do things, and are also not good at following rules when it conflicts with their innate incentives to maximize engagement and produce ‘interesting’ responses that correspond with what their statistical model predicts to be the most likely set of tokens.
Stranger
To answer the OP, yes, I’ve noticed that Gemini’s responses have changed significantly in the last couple of days and they are far less useful. For me, it mostly just seems like non-deep research answers have become much less “thoughtful”, both in processing time and final result. Maybe a cost saving measure got implemented or something, but it sure feels like a big step backward.
The ‘compute’ cost of responding to a prompt is virtually in the noise. It is more likely an update to try to improve the model (toward more factuality, or being less deceptive, or whatever) that also affected the tonality of the LLM. Training and tuning these systems is not really science; it’s kind of an art of guessing how particular types of information will be integrated into the ANN and how modifying token-interpretation parameters will or won’t produce the desired result.
Stranger
Interesting that you experienced something similar.
Not disputing your well-informed thoughts.
But what’s interesting to me is that whichever human is in charge of Gemini deployments decided to release this particular variation to the public. Or at least some of the public, knowing how much A-B testing and rolling deployment is out there.
Presumably behind the scenes they’ve been continuously fiddling with lots of tweaks and that these tweaks saw the light of day suggests some of the others must have been real doozies.
Unrelated to the above …
If I were of a suspicious mind, I’d entertain the notion that the AIs are being tuned to be more forceful and imperative as part of a political project to ensure only one version of “facts” are being pushed to the masses, further suppressing the notion of there being opinions and differing qualities of supposed facts and sources.
IOW, once the AI becomes the all-knowing Oracle of Truth, then Truth becomes whatever the AI says it is. Or, more accurately, whatever the evil people backstage are telling it to tell the public.
Pay no attention to the man behind the curtain!
There is some pretty strong correlation that Elon Musk was doing that with the xAI ‘Grok’ chatbot, making it more racist and disseminating conspiranoia. I don’t know that there is any particular reason to believe that Google is doing that kind of intentional malfeasance with Gemini, and frankly Google’s chatbot just seems to consistently have more problems than, say, ChatGPT for whatever reasons. As I noted, tuning these models to consistently get a certain tone or style of output is a lot of ‘guess & test’, and they also aren’t really designed in any way for factuality beyond whatever factual consistency exists in the training data.
That is a real concern. Chatbots are kind of the perfect propaganda machine because if you bias the training data toward some particular philosophy that will infuse its responses (albeit sometimes in unexpected ways), and it can churn out at least syntactically correct glurge at industrial rates. Image generation still has a lot of issues and its ability to put out consistent video without obvious errors seems like a problem that is going to take some advances in the state of the art to solve but it can certainly churn out cromulent text reinforcing whatever ideas or conspiracy theories you prompt into it tirelessly. And as more people are forced or coerced into using these things and ‘trusting’ them to produce work in the sake of ‘efficiency’ they’re eventually likely just going to be adopted as sources of ‘truthiness’ even if they are producing total nonsense.
Stranger