Large language model AI doesn't learn

Great Debates
21

I’m not an AI specialist at all, but the field is partially related to linguistics, which I’ve learnt a bit about.

I’ve just finished a Malagasy language MOOC. Some of the lessons were dedicated to theory, mainly grammar points. Unfortunately, the examples given were rarely translated. So I found myself in the weird situation were I was aware that I repeating a sentence that contained a verb in the, say, the future form. Or, that I was asking a question. I could recognize the grammatical markers and analyse the sentences as “future” or “question”. But because of the lack of translations, I had no idea what I was saying most of the time.

Do I feel I “learnt” something and gained "knowledge that I didn’t have before ? Definitely. Did I “understand” this new “knowledge”. Not fully, far from it.

22

Yes, the difficulty I find with this debate is that it focuses on what an AI is doing in order to assert that it’s not learning.

But I think the real question is that we lack clarity on what exactly humans are doing that is qualitatively different, if anything.

23

For one and two, the answer, in my experience, is just “no.” I’ve never seen an answer repeated verbatim. Sometimes, the same themes may be struck, but I can’t remember getting anything word-for-word.

For the last question, I do believe there is a random seed used to generate the answer, and any context the LLM may already have in the conversation will shade the result one way or another.

Whatever AIs do, I’m perfectly fine calling it “learning” and not thinking much more about it. Close enough for me. And I love it. Today, I asked it to make a little vocabulary quiz game for me in Hungarian and it did it lickety split, all accurate. It did infinitive verbs. It did nouns when I asked it. I asked it for short, familiar phrases and it quizzed me. Then I asked it for mnemonics to help me remember verbs that don’t quite stick to me, and it did that, too. The damned thing is magical.

24

@Moonrise has touched on the “qualitatively different” bit.

In college, I once worked as a writing tutor wtih a geology student who was there against his will and who resented me greatly. It didn’t help that he was writing with highly technical language, and I didn’t recognize about a third of his nouns, verbs, and adjectives. But I could tell that he was structuring his sentences terribly and often in ways that broke basic rules of standard English–run-on sentences, nonparallel structure, and the like. So I was able to advise this angry dude on how to clean up his writing, even though I had no idea what the content of the writing was.

I used my senses to absorb his writing. I applied a set of algorithms to the writing. I changed and improved the writing. But I had no idea what the content was. I learned nothing from the experience, except possibly a lesson about how grammar and meaning can be thoroughly separated.

Large-language models are a lot like that.

25

I kind of feel like maybe the OP should take one of those “Intro to Machine Learning” online courses to get a better feel for how these things work.

They’re not “learning” in the artificial general intelligence way of thinking. What they’re doing is much more like what @Max_S describes- basically someone defines a set of criteria that define the model, and then “train” the model by feeding it a whole bunch of data that’s been scored or whatever, so that the model can determine whether something is or isn’t whatever it is.

So for example (dramatically simplified), someone may define the model as being a set of patient data with a specific set of diagnostic exam results to examine for a specific condition- say probability of having cancer in 5 years. Then they’ll feed the model thousands of diagnostic exams for people over time, with the ultimate result of whether or not they had cancer. The model will “learn” how to identify those patients whose exams indicate that they’ll have cancer in 5 years. “Learning” in this case means that the model identifies the set of criteria that actually identify those patients on the existing training patients, and can apply those criteria to future ones.

It’s that identifying of the criteria and their weighting/importance automatically by the model that’s the hard part to understand, and the part that’s the most opaque from the outside. It’s not “learning” in the usual sense of the model understanding it- that would be artificial general intelligence. But it is “learning” in the sense of the model determines its own criteria and weighting for identifying things, without human intervention, except insofar as the training is concerned. We don’t know if it’s looking at age, vaccination status, previous diagnoses, etc… when it’s determining if someone will get cancer in 5 years- all we know is that we fed it a lot of exams and patients and told it which ones did get cancer in 5 years. It’s looking at all that and coming up with its own model to extrapolate on future patients and figure it out.

26

Try working with it to develop a “new” language. I found it pretty insightful:

27

Here you appear to be getting into the metaphysical idea of whether an AI’s understanding of words has the same kind of connection to real-world objects as a human does. As endlessly discussed elsewhere, I don’t happen to think this is a productive line of argument, but in particular, it has no bearing on what we mean by “learning”. I think for purposes of this discussion it’s more useful to take a straightforward dictionary definition of “learning” as “the acquisition of knowledge or skills through experience, study, or by being taught”, meaning quite simply that it’s a process that modifies behaviour, typically in an enhanced and productive way.

To answer this question about LLMs, we have to narrow down some specifics: which LLM in particular, and by “learning” do we mean pre-release training or do we mean continuous learning in the wild?

Let’s narrow down the first part by picking ChatGPT as our example, because it’s currently the most prevalent and talked-about. Can ChatGPT learn? Yes, it has undergone very extensive and very computationally expensive training that included both supervised learning (to help it generate more human-like responses) and reinforcement learning from human feedback (to help it recognize more preferential responses). Indeed the core of ChatGPT is a multi-layer neural network where extensive training is the primary paradigm for achieving its functionality.

The second question is whether ChatGPT continues learning in the wild. The answer is sort of yes and no, but mostly “no”. No, its pre-release training creates a static entity, along with its fixed (although enormous) dataset. Differences in responses to the same input from one session to another are just randomizations. It might be said to “learn” in a very narrow, time-limited sense because it retains the context of the conversation in any one session, so you can go back to it and ask it something related and it knows the context. More significantly, perhaps, if you ask it to solve a problem and it gets it wrong you can give it hints and corrections and you can (often, not always) nudge it to the right answer. Whether that can properly be called “learning” is a matter oif opinion, and it only persists within the one session.

But without a doubt, ChatGPT’s performance is almost entirely due to learning from extensive pre-training (and is what the “P” in GPT stands for; the “T” is for “transformer”, a type of neural-net based deep learning model).

28

I am far from an expert, but if you agree with this, that’s what I’m saying. The word “learn” is deceptive, and it leads to a lot of confusion. For example:

It’s not insightful. It might aid your insight, but it’s not insightful. It’s a tool, not a second mind to bounce your ideas off of.

Again, characterizing LLMs as “learning” is part of a trend toward anthropomorphizing them, and that anthropomorphizing leads to errors in how we relate to them.

A new word would be helpful. I’m not convinced that “machine learning” as a phrase is sufficiently distinct from “learning” that it’ll help people avoid that misguided anthropomorphization.

29

One things that’s interesting to me in this thread: the OP of the thread is centered around the concept of “schema.” If you don’t engage with the concept of schema, you’re not really engaging with the OP. Yet that word doesn’t appear anywhere in the thread except in my posts. A lot of folks appear to be ignoring the OP in favor of responding to the title–which is your right, but I think it misses an opportunity to discuss one of the central ideas in how humans learn.

30

Classifying chatGPT as an LLM is incorrect and probably the basis for the confusion. It is, technically, a:

Effectively, it’s like a set of neural nodes glued to an LLM.

It’s akin to our system of seeing where the brain doesn’t receive pixel data from the eyes, the information goes through a compacting/boundary finding system first so that the brain can deal with larger chunks of information and do a better job of handling it (in most cases).

Strapping an LLM allows the neural net to handle language in a way that’s easier to think upon because it has a recommended template to go from at every stage.

It’s definitely “trained”. You can get a job as a trainer, in fact. Now whether you personally feel like that’s equivalent to training a human or an animal is up to you - training is the official, technical word for the activity. I’d say that we don’t currently have any reason to think that a human brain is any different than a neural net strapped to some specialized hardware that makes it easier to do what we do. It’s safest to assume that it’s learning like a damaged human that lives in a box and doesn’t understand things because it has never been outside of the box.

If you fix the box issue, you’ll still need to deal with the damaged part. We don’t know what’s missing from it that we have, yet, but it does basically work like us otherwise. IMHO.

31

This works fine for humans, but it breaks down when we talk about machines. When I point a Polaroid camera at a page of text, is it “studying” the page of text? Is my act of pointing the camera an act of teaching? When the image of the text appears on the film, has the camera acquired the knowledge? When it spits out a picture based on that text, has it modified its behavior according to the knowledge that it has gained? Is any of this changed if I rig the Polaroid to get a double-exposure, such that its eventual picture is a melange of two different scenes?

When I point ChatGPT at a page of text, is it “studying” the page of text? Is my act of pointing it at the page of text an act of teaching? When that data appars in the database, has ChatGPT acquired the knowledge? When it spits out a product based on the text, has it modified its behavior according to the knowledge that it has gained? Is any of this changed if I feed it a million pages of text, such that its eventual product is a melange of a million different pages?

I just don’t think that definition works well with machines in their current state. It’s a starting point, not a deep dive into learning.

32

I apologize that I was unclear in the OP. I know that words like “learning,” “training,” and “neural” are used to describe these tools. Telling me that they’re used isn’t productive, because I am specifically disagreeing that they are either literally correct or metaphorically wise.

That’s a fascinating analogy–but even there, I’m not convinced it’s accurate, except for extreme values of “damaged.” ChatGPT has a set of algorithms that allow it to do a lot of stuff, but the basic act of understanding what it’s talking about eludes it. That appears to be a fundamental, not an incremental, difference between how it works and how a human mind works.

It lacks schema.

33

I guess the way I’d describe it is that it’s somewhere between normal learning like a human or animal does, and having a computer just compare to some kind of predefined criteria.

It’s definitely a lot more sophisticated than say… having your computer just look at a predefined set of diagnostic criteria like say… blood pressure, cholesterol, and blood glucose, and then spit out a list of patients who are at risk because their values are above some specific threshold.

The machine learning version might look at 10,000 patients and their outcomes, and determine on its own that the relationship between age, BP and blood glucose is the determining factor, and if it’s above a certain amount, that patient is at risk.

I’m not sure how it works under the hood- I’ve taken a couple of those “using AI” type courses, and set up my own models, etc… and they just basically teach you how to set up the model and train it, but don’t really tell you how the model is determining internally what pictures are rain clouds and which ones aren’t, based on your training images. (that was one course example I did). It might be shape, it might be contrast between cloud & sky, it might be both; nobody really knows. But it’s awfully good at it, I’ll say that.

Machine learning is fantastic at identifying things- that’s why it’s so useful for things like parsing spoken speech; if you feed it a whole bunch (tens of thousands) of examples of spoken language and the corresponding words for each example, it’ll learn exactly what defines those words when spoken and identify them very accurately. It won’t know what the words mean though.

Something like ChatGPT is just a very extended version of that- it’s been trained on thousands of examples of written samples - essays, newspaper articles, literature, etc… and been told a lot of information about each- type, subject, etc… and a whole lot more I suspect. Same with the art generators- thousands of pictures and photos, and information such that if you say “in the style of Caravaggio”, it has some kind of data about photos that define what that means visually.

34

So what, exactly, is “understanding” in a human brain?

I gather by “schema” you meaning clustering of concepts, testing provisional hypotheses about the relationship between concepts against data and updating them? I’m not sure what you’re getting at with this. It seems to me that something like ChatGPT does this rather well.

35

I still find this older article from IEEE Spectrum useful (almost 2 years old! An eon in AI generations).
7 Revealing Ways AIs Fail - IEEE Spectrum

From the article:

Part of the problem is that the neural network technology that drives many AI systems can break down in ways that remain a mystery to researchers. “It’s unpredictable which problems artificial intelligence will be good at, because we don’t understand intelligence itself very well,” says computer scientist Dan Hendrycks at the University of California, Berkeley.

Here’s a more recent piece that is more of an opinion piece, but from an engineer on his third robotics startup, so someone whose opinion has some expertise behind it.
Just Calm Down About GPT-4 Already - IEEE Spectrum

From his interview, a couple of interesting thoughts:

I’ve been using large language models for the last few weeks to help me with the really arcane coding that I do, and they’re much better than a search engine. And no doubt, that’s because it’s 4,000 parameters or tokens. Or 60,000 tokens. So it’s a lot better than just a 10-word Google search. More context. So when I’m doing something very arcane, it gives me stuff.

But what I keep having to do, and I keep making this mistake—it answers with such confidence any question I ask. It gives an answer with complete confidence, and I sort of believe it. And half the time, it’s completely wrong. And I spend 2 or 3 hours using that hint, and then I say, “That didn’t work,” and it just does this other thing. Now, that’s not the same as intelligence. It’s not the same as interacting. It’s looking it up.

It gives a really good technical understanding. What the large language models are good at is saying what an answer should sound like , which is different from what an answer should be.

36

I posted this example elsewhere, but it fits better in this thread.

I just read another teacher’s list of Chat-GPT-created jokes around exams, including such gems as " Why did the state testing room get so hot? Because all the students were ‘testing’ their patience!" and “Why did the English test go on a diet? It wanted to lose some unnecessary words!”

Both of those are awful, but I want to look at the second one in more detail. It’s like a Viceroy butterfly. Viceroys look like Monarch butterflies, but only if you don’t look closely; and they’re completely non-poisonous, unlike monarchs. This looks like a joke, but only if you don’t look closely; and it’s completely non-funny, unlike jokes.

It looks like a joke because it follows a traditional riddle structure:

  1. It starts with a “Why” question.
  2. The subject and the verb are a strange juxtaposition that you wouldn’t encounter in everyday life.
  3. The punchline references elements from the subject and verb in a sentence that sounds reasonable.

It’s lacking any sort of wordplay in the answer that would make it funny, though; and ChatGPT doesn’t add wordplay because it doesn’t know what wordplay is. It doesn’t know anything, of course–this is just one subset of not knowing anything.

So I was trying to think how I would make a similar joke. I started by thinking, “How could a punchline about going on a diet involve wordplay?” and came up with the word “pounds,” which has multiple meanings. It can mean money, or it can mean hits. A punchline could be, “Because he/she/it wanted to shed some extra pounds,” as long as there’s a double meaning to that sentence. That’s close enough to ChatGPT’s attempt that I think it’s comparable.

My first attempt was something like, “Why did the British shop go on a diet?” But the punchline doesn’t work, because I know that shops don’t want to shed pounds, they want to gain them.

Then I thought about boxers. “Why did the beat-up British boxer want to go on a diet?” Again, though, I know that “shedding extra pounds” is a really bizarre way to describe not getting hit, so bizarre that most people wouldn’t get what it meant. So I rejected that as well.

What if I switched it around? Instead of “Why did X go on a diet?” I make it about a diet person wanting to do something else? I tried, “Why did the British diet guru buy something?” That works better, but it’s still pretty confusing: when you’re buying something, you do “shed pounds,” but it’s not what you want to do, so the punchline doesn’t really work.

Spending spree! That’s where you get rid of a lot of money quickly! And I finally got my joke:

Now we have a joke that has a question with an unusual juxtaposition of subject and verb, and a punchline that works for both aspects of the question, based on wordplay.

This is not a great joke. I don’t need you to tell me that; I know that. But it is a joke. It’s a monarch, not a viceroy. If it showed up on your Laffy Taffy wrapper, you wouldn’t be like, “What the fuck was that?”

What’s interesting to me is my thought process:

  • I had to break down what I knew about a specific kind of joke structure.
  • I had to search my brain for diet words that had multiple definitions.
  • I had to think about what I knew about those different meanings, and what I knew about the world (e.g., that shops don’t want to lose money).
  • I had to reflect on idiomatic usage of words and to predict how people would respond to a particular phrase.
  • I had to abandon a structure and reverse it.

In the end, I had a joke, and I can grade it as a C- or D+ joke.

That’s fundamentally different from how Large Language Models work.

It is possible that eventually we’ll get an AI that can make real jokes, not pretend jokes, but I don’t know how that will work, as long as the AI can’t engage in the sort of reflection that any human can engage in. Maybe brute force will eventually enable an AI to make real jokes; but barring a major change in process, it won’t ever be able to engage in reflection.

Incidentally, I propose we use the word “viceroys” to describe the sort of AI products that look like the real thing but are missing a critical element.

37

Instead of thinking how you would make a similar joke, what about a very young child, who’s still learning how jokes work?

38

I think because your opening paragraph is not well answered by the concept of schema.

Just as it’s impossible (for another human brain) to understand the complete internal state of a human brain, it’s also impossible to understand the complete internal state of a sufficiently advanced computer system. We do have many versions of a “theory of mind” that we apply to each other, with varying degrees of success. I am even more skeptical that such models are useful for computers.

Thus, for me, the whole “let’s see if we can apply this particular theory of mind to that type of computer system” is rather a dead end. It’s not amenable to falsification. If that’s truly all you want to discuss, that’s fine of course, and I’ll bow out.

Instead, the discussion that I think is interesting is how we as humans interact with computers that are seemingly similar to us. From your opening post, I thought your comparison of how some things learn while others do not was instructive. That opens up definitional questions like “what do we mean by ‘learn’?”, “what do we mean by ‘knowledge’?”. And then from there mechanical questions like “what does learning look like?”, “how can we tell if something was learned?”, “what’s the difference between learning and other changes?”, etc. There’s a lot of observables and clarifications of terms that could be discussed without worrying about incomprehensible internal states and unverifiable theories of mind.

39

You appear to be influenced by the same bias that Marvin Minsky talked about years ago – that because the entity under discussion is a machine, and because we have a rudimentary understanding in principle of how it works, that it therefore is not capable of “real” intelligence nor of “real” learning, but is only mimicking those behaviours. That may actually sometimes be true, but at some point one has to draw a line and acknowledge that it has been crossed.

The essential difference between an AI like ChatGPT and your Polaroid analogy is that ChatGPT synthesizes and generalizes information, and can thus produce new information and potentially establish new relationships between facts – all hallmarks of intelligence. This, in fact, is closely analogous to your “schema” concept in children’s cognition, certainly more so than a Polaroid picture or molding a piece of plastic.

To give a very simple example, the other day I was reading a medical research paper that had a paragraph I was having trouble understanding because of the medical jargon and phraseology. I typed it into ChatGPT and asked it to give it to me in layman’s simple English, which it did, and as it turned out, quite accurately. It may or may not have had that paper in its vast database, but it surely didn’t have the simplified translation, and I’m sure that neither ChatGPT’s programmers nor its trainers had ever seen it before. So tell me, where did that simplified translation come from?

Also pertinent here is that prior to training, it would certainly not have been able to perform this task at all, a fact also evident in other impressive AI systems like IBM Watson, which eventually acquired super-human skills in playing Jeopardy.

There is no reason to believe or require that an AI’s cognitive processes should necessarily mimic those of a human, as long as its responses meet our expectations of intelligence and learned behaviours. Nevertheless, as I and @Riemann have mentioned, ChatGPT’s ability to synthesize fact-relationships is much like the concept of a schema.

All of that aside, the concept of “learning” in AI is really a pretty basic and perfectly valid one. It essentially means that the creation of an AI is compartmentalized into the development of one or more programmed frameworks, such as a set of layered neural networks, and then a set of training exercises that teach the system task-specific behaviours and subject matter knowledge. Neither programmers nor trainers have any a priori knowledge of how the resultant system will behave, and refining its performance generally requires repeatedly fine-tuning its training.

40

Yes, this is a great one-sentence statement of what I’m trying to get at.