I think training was used in the sense of training a dog, not the ML sense of training model weights, but I could be wrong.
I’m not sure exactly how OpenAI is implementing the memory feature, but there a couple of methods, including RAG (retrieval augmented generation) and prompt injection.
I’m having a hard time parsing that bizarre statement for any sort of meaning.
In any case, the truth of the question hinges on what one means by “paying customer”. In the current experimental model, a paying customer gets only the extra amenities as described. In the future, however, OpenAI may choose to offer customized versions of GPT to corporate clients that are trained on specific subject matter, and these systems may be continuously or periodically tweaked to enhance performance or provide the latest knowledge.
Such systems are likely to have deeper subject knowledge and greater accuracy than today’s GPT. Subject-specific training is how IBM has been marketing Watson, its intelligent query engine. Its success as a Jeopardy contestant was due to extensive training for that specific task.
Literally wasn’t responding to you, mate. I was responding to someone else describing present-tense actuation.
Watson was a tremendous business failure.
To the best of my understanding, that’s correct. One of its greatest expected potentials, being an advisor to physicians on complex diagnoses, was probably its worst failure.
But there are at least two things to keep in perspective. One, not everything IBM Research produces is necessarily best-in-class. IBM’s mainframe computers certainly never were. Their token-ring LAN technology intended to be a superior version of Ethernet was also ultimately a failure in the marketplace. But today we have LANs everywhere.
The failure of a particular technology implementation is not a useful critique of the broader concept of the technology itself. And “AI” isn’t even a “technology” per se, it’s just a description of a particular class of advanced computing protocols that exhibit apparent intelligence.
May I suggest that your infatuation with AI is due to LLMs’ mastery of “proper grammar” and that you use it only as a toy? I’m curious if you’ve considered that?
You can suggest whatever the hell you want. My older brother was involved in AI research. My visits to him at his several sabbaticals resulted in opportunities to speak with some of the leading researchers in AI at the time, including Marvin Minsky at MIT, John McCarthy at the Stanford AI Lab (SAIL), and other researchers at places like Xerox PARC, as it was known at the time. My brother himself was not an AI researcher, but professionally more interested in human cognition, and quite well known for his research.
So no, my understanding of the future of AI isn’t based on the fact that ChatGPT, like, talks good. 
ChatGPT supports fine-running (at least for some of their models). It’s a paid service based on the training and eval dataset sizes.
Fine-tuning works well in many cases, but it’s a pain to create the dataset and for many tasks it is a bigger pain to figure out the evaluation methodology.
https://platform.openai.com/docs/guides/supervised-fine-tuning
Yup, it’s one of their key business uses. Have you gotten to work with it? Evaluation does seem like an important question.
I have not done ChatGPT fine-tuning. We do LoRA fine-tuning on our own LLMs at work. For tasks that have a clear factual answer (classification, formatting, etc.) it is pretty straight-forward. But for tasks that are subjective (summarize, rewrite, outline, etc.) it is difficult to objectively measure the evaluation. These are on small LLM’s so we want to wring out all of the quality we can.
With large, cloud models one-shot and few-shot examples work very well for these same tasks.
This is brilliant.
In the last few years it’s been striking to me that while AI “art” and “music” and math (as a stand-in for factual content) are all on the iffy-to-babyshit spectrum, AI poetry is generally pretty impressive. Meter, rhyme, even implicit meaning! My hypothesis was that we have to bring a lot of ourselvs to poetry, so slop is almost a boon.
It’s wild that poetic rather than prosaic prompts can elicit explicitly prohibited content — almost like the hack goes both ways. Thank you for this line of discussion, @stanislaus!
Unfortunately, I don’t think you can read this if you aren’t on bluesky, but this whole thread (about the deterioration in effort and skills in their college students, not just inbound, in this chatbot era) is something. Something bad, specifically.
I didn’t say anything about your understanding. It’s obvious that your irrational exuberance about the current capabilities of AI stems from the fact that LLMs talk good.
Your posts betray no understanding.
That’s downright Presidential.
Lots of things are “obvious” to those with pre-existing beliefs.
I’m well aware that back in the 60s and 70s there was indeed unfounded optimism about the near-term future of AI. It took a long time to get us to where we are today, which is inestimably far from perfect and certainly nowhere remotely close to AGI which we may never achieve just because there’s no apparent need for it. And the LLM model is certainly not the be-all and end-all of AI technology – just one interesting aspect of it that will no doubt continue to be useful.
Your description of my so-called “Infatuation with AI” is a mischaracerization of the fact that LLMs are a niche area of AI that’s produced unexpectedly impressive results. Yes, it’s easy to be misled by AI that simply produces grammatically correct sentences, but it’s a lot more difficult to explain LLMs that have evolved emergent problem-solving skills that many humans would not be able to solve. And could be solved by exactly 0% of Trump voters. The world would be a far better place if ChatGPT, for all its faults, could vote.
Can you provide specific examples of emergent problem solving skills that AI researchers have been unable to explain?
This is a bizarre statement.
Yeah, it’s a bit out there, but have you ever listened to an interview with a typical Trump supporter? Ask them to name the three branches of government. They can’t. GPT can. Ask them about the importance of checks and balances between the branches. They won’t know what yoiu’re talking about. GPT will. Ask them about the policies of their state senators and Congressional representatives. They won’t know, or even know who these peoplel are. GPT will.
All I’m saying here is that decisions made on the basis of accurate analysis of available information generally yield better outcomes than decisions based on ignorance. I’m certainly not trying to ascribe moral values to GPT, which would be silly.
Give me a specific emergent ability that cannot be explained (which is different than “predicted”). And preferably more recent than 3 years ago.
There was a big hubbub when ChatGPT was released publicly (around the same time as the article above) about unexplainable abilities. Since then, more rational minds have pointed out that those are easily explainable, and in many cases are predictable as well.
Maybe there’s something I’m not aware of that still cannot be explained. But to the best of my knowledge, that “unexplainable” thing was breathless hype by people who didn’t know what they were talking about, or by media who misinterpreted what knowledgeable people were saying.
2025 Emergent Abilities in Large Language Models: A Survey (Berti, Giorgi & Kasneci, 2025) Emergent Mind+1 A comprehensive survey that: reviews definitions of “emergent abilities”, summarizes which capabilities have been reported (reasoning, in-context learning, code generation, problem solving), and critically examines under what conditions these appear (scaling, pretraining loss, prompting, quantization, etc.). Emergent Mind+1
2025 LLMs and Emergence: A Complex Systems Perspective (Krakauer, Krakauer & Mitchell, 2025) Emergent Mind Frames emergence in LLMs through a complex-systems lens. Examines whether emergent capabilities reflect genuine emergent intelligence (with internal coarse-grained representations), rather than just scaling artifacts. Provides conceptual clarity about different kinds of “emergence.” Emergent Mind
2025 Emergent Abilities of Large Language Models under Continued Pre‑training for Language Adaptation (Elhady, Agirre & Artetxe, ACL 2025) ACL Anthology+1 Empirical work showing that emergent abilities can arise (or shift) when LLMs undergo continued pre-training (CPT) for language adaptation — even when the original model was English-centric. This speaks to the dynamics of emergence under distribution shift. ACL Anthology+1
2025 Emergent Response Planning in LLM (Dong et al., 2025) arXiv Presents evidence that LLMs’ hidden representations encode “future outputs beyond the next token” — structural and content attributes of full responses — suggesting a form of emergent planning behavior. This challenges the simplistic view of LLMs as only “next-token predictors.” arXiv
2024 Understanding Emergent Abilities of Language Models from the Loss Perspective (2403.15796v3) Emergent Mind Rather than tying emergent abilities strictly to model scale, this work studies them through the lens of pre-training loss: it shows that models with similar pre-training loss — even if different in size — can have comparable downstream performance, indicating that emergent abilities may depend more on loss dynamics than model size per se. Emergent Mind
2024 Are Emergent Abilities in Large Language Models just In‑Context Learning? (Lu et al., ACL 2024) ACL Anthology+1 A critical study challenging the emergent-abilities narrative: through extensive experiments, argues that many purported “emergent abilities” may be explained by in-context learning, model memory, and linguistic knowledge rather than some scale-driven jump in capability. Raises caution about overinterpreting “emergence.” ACL Anthology+1
Does this really support your assertion, DrWolfpup?
It supports the assertion that emergent properties are not well understood and there are conflicting explanations for them. The most commonly accepted hypothesis is that they arise as a function of scale, specifically, the size of the ANN and the number of parameters, which in GPT-5 are believed to number in the trillions.