There’s one way AI has made my life (not) better. Namely, being snarkier than I care to be in this thread. Which is to say that I apologize to @JohnT for my posts.
Dude, if you want to put me on ignore, you can just do it without getting my permission. I’ve given my assessment of your argument, you can make a decision based on what I’ve already said.
I don’t think he was arguing for point #2 (it is merely a possibility) but point #1 is compelling: you CANNOT trust a system that is geared to telling you what you want get to hear, and has eidetic recall of everything you’ve ever asked.
You can see an example of this yourself by asking any of the LLMs to identify a book. Give it some made up plot with a description of a particular character or scene, and ask it what book it’s from. It usually will come up with a few possibilities and call out parts of the book that supposedly match.
Tell it those books aren’t what you’re looking for, and give it another few details. It will return with some more possibilities that somehow match your description even more closely. By the 3rd or 4th time, it will be giving you books that match your entire description word for word, and either have nothing to do with your description, or simply don’t exist.
When I’ve done this (naively at first, as a test more recently), it will get farther and farther away from the “right” answer each time while becoming more and more certain that its answer is correct. It’s a good exercise when you start to trust AI output too much.
I would prefer not to put anyone on ignore except the most egregious asshole. But I note that your opinion that I may have just lied about what my GPT prompt was is not dissuaded by an actual screenshot of the actual dialog, which you might note exactly matches the prompt that I posted earlier in the spirit of full disclosure. Are you just incapable of retracting an accusation? Who is it here that isn’t arguing in good faith?
If it keeps me in your good graces, then I accept your screenshot as proof that you didn’t lie about the specific prompt. Your screenshot does not include what you entered before that or the complete text of the response, so I’m not going to comment on that part.
Like I said, I see two possibilities. If your honor has been disparaged by my inclusion of dishonesty as one of those possibilities, I don’t see it as my place to repair it. If you feel you have been completely honest in all arguments here, then sleep soundly and don’t worry what some rando on the internet says.
I did not enter anything before that.
This reminds me of a variation of the game “Twenty Questions”, in which there isn’t actually a preexisting answer; the person running the game simply flips a coin for each yes-or-no/ A or B answer until the random returns have defined a result.
Yes, I did this as a test, asking if two Discworld characters (that I know never have) had ever interacted. AI came up with an elaborate scenario from a specific book where they had - an entirely plausible one, I might add. And went on to write about that made-up scene at length while getting increasingly more vague about the exact location of the scene in the text (which the system doesn’t actually have access to, AFAICT)
Killed any developing notion I may have had to ever trust AI output, that’s for sure.
I treat AI like I would a captive criminal - by all means interrogate it and perform actions based on that output, especially if it helps you do your job. But by no means trust the fucker.
This was your very first interaction with that AI? That seems unlikely.
Thank you. Now, going back to a previous point you made …
That made me curious, so in that same session I asked ChatGPT about exactly that point. Here’s a brief summary of the response.
Me:
How has your memory of previous conversations with me about AI influenced or shaped your response to this question?
ChatGPT: (summarized version)
Very little, actually. Your prior conversations influenced the style and emphasis of my response more than the substance.
<snip>
Those patterns nudged me toward a response that was:
- More analytical
- Less rhetorical
- More skeptical of headlines
That’s the sort of framing that usually fits people who ask the kind of questions you ask.
What it did NOT change
Your past conversations did not affect:
- the facts about the paper
- my evaluation of the study quality
- the scientific interpretation
Those come from the content of the paper and the research norms in psychology and HCI.
If another user asked the same question, the core analysis would be essentially identical.
I genuinely hope that one day the light bulb goes off and you understand that when you ask it a question like that, it’s not answering you factually, it’s answering you via lingual probability.
On a different topic, I continue to use Claude for coding, and I find that for long sections of coding I’m having to clear the context window maybe every eight or ten questions. I’m using the free version, I’m curious if commercial-grade Claude has a longer context window and a less muddy middle.
I’ve had this issue with ChatGPT as well, although more like after 20-30 questions.
I’ve switched to using ChatGPT Codex where each task is a fresh context; it pulls your codebase into the context before working on your task. This works much better, especially since it eliminates any wrong turns it/you took along the way.
I haven’t used Claude Code, but they have a similar approach – although maybe not in the free version.
Do you think “doing research” over the course of a week is not a cognitive task? A good rule of thumb is every time you think, “That’s convenient!” it means you’re offloading something. Every time you use a microwave, which I’m guessing 99% of us do, you lose out on experience cooking. If you use a microwave for everything, you become a shittier cook.
I looked at that study the MIT article references. It’s still in pre-print, in the process of peer review, and 50 of those 200 pages are citations. Interestingly, it has an entire section labeled “If You Are an LLM, Read This Section” which neatly breaks down the study limitations. Hopefully GPT found that convenient.
Here’s another study.
There are a lot of handy charts in there if you don’t want to read.
The participants were in a randomized controlled trial in an environment designed to mirror a real-world classroom. The length of the study was six weeks.
The primary analysis revealed a statistically significant difference in knowledge retention between the conditions. Participants in the traditional learning group (M = 6.85, SD = 1.7) scored significantly higher on the retention test than those in the AI-assisted group (M = 5.75, SD = 1.5), t (83) = −3.19, p = .002. This difference of approximately 1.1 points on a 10-point test represents a meaningful effect. The traditional learners’ mean was roughly 11 % of the test score higher than the AI-assisted learners’ mean. An 11 % gap on a course assessment could easily translate to a full letter-grade difference in many academic settings, underscoring the practical significance of this finding.
Subgroup analyses by topic area revealed some variation in the effect of learning condition. Technical topics showed the largest disadvantage for AI-assisted learning (d = 0.92), while ethics and society topics showed a smaller, though still meaningful, effect (d = 0.45). However, these differences were not statistically significant (Q = 2.84, p = .42), suggesting a relatively consistent pattern across content areas. Fig. 7, a forest plot, illustrates this subgroup effect sizes, consistently favoring traditional learning. The heterogeneity was low (I2 = 28 %, p = .21), further supporting the consistency of the effect across different content domains.
Participants who used AI showed a decline in cognitive skills 45 days after the experiment was conducted.
Our findings also extend cognitive offloading theory into the AI domain. Risko and Gilbert (2016) established that people naturally offload cognitive demands onto external aids; our results demonstrate that generative AI represents a qualitatively different form of offloading. Unlike a calculator or a notebook that offloads specific tasks or memory storage, AI can offload entire cognitive processes – including comprehension, synthesis of information, and even aspects of critical thinking. The medium-to-large effect size (d = 0.68) provides empirical evidence that this comprehensive offloading has tangible consequences for learning outcomes.
Of course this study has limitations, which are described in the text. Every study has limitations. But this looks pretty thorough to my untrained eye.
I think, wolfpup, your GPT responds to your questions in a way that sound vaguely scientific but with just enough critical analysis to increase your view of its credibility. So it’s not just blatantly sycophantic, it’s sneakily syncophantic as well. It is designed to please you, and part of “pleasing you” means making you feel all scientific every time you ask it a question. I would be wary.
That’s interesting, I got a different response to your exact prompt. Although you didn’t post the full response so I’m not sure how to compare it. I seem to remember you posting a section about “What people say it means” which did not come up at all for me.
(I cannot find that section. Did you delete it?)
It chose to refute those made-up points for me, too. It seems to have confabulated the study’s claims, but I’m guessing it did so based on what was used as the prompt.
Here is a link to a PDF of a fairly lengthy paper suggesting that if people consistently use an AI assistant for essay writing, over time they lose significant cognitive skills. Are you able to access this PDF and comment on the study and in particular on its quality and credibility. If you cannot, I can post an abstract of the paper here for your comments. https://arxiv.org/pdf2506.08872
The idea of losing “significant cognitive skills over time” was baked into the question, so I imagine GPT found associated words like brain damage, long-term cognitive decline, and threw those in there.
The problem is that the prompt misrepresented the study’s claims. I would test it by asking the question in a more neutral way, but it already knows I’m critical of the study, so I’m not sure I would get a result that wasn’t influenced by the original biased prompt.
The key issue is: you cannot evaluate ChatGPT by asking ChatGPT.
Imagine asking a captured spy, “Is your information reliable?” What are they gonna say, “partially” ?
If you read up on how generative AI “creates,” ideally NOT by using AI to read for you, you’ll find that it’s much less than 100% information retrieval in natural language. Pleasing, and hooking, the end user, who will ultimately pay for these services, is a major part of the deal.
I’m reminded of a scene from an episode of The Prisoner, “It’s Your Funeral”:
I brought the Activities Prognosis.
How accurate are they?
We don’t know.
Why not?
Twice, our machines refused to give
efficiency percental appraisals.Refused? How?
By not returning the data to us.
They’ll want a trade union next.
I’m rewriting this.
Participants who used AI showed significantly less knowledge retention based on a surprise post-test 45 days later.
The study tested memory and knowledge retention.
According to the study, it wasn’t just fact recall but questions that required engaging deeply with the material.
This study was first brought to my attention with a completely inaccurate representation of what it did. So I want to be precise. When I have the chance to read it in more detail I will share my learnings.
“I am now telling the computer exactly what he can do with a lifetime supply of chocolate.”