That data centers are worth building shows that the problem scales. The objection seems to be that the solution doesn’t scale linearly, but hardly anything does. Yes data centers use lots of resources, perhaps too many, but that has nothing to do with scaling.
I worked on parallel computation when it was new. Here is an example of something which does not scale. Fault simulation is the process of running tests on a simulated digital circuit, inserting possible faults into it, and see which ones get caught. There are a lot of faults, so we hoped that if we could split the fault across a small bunch of computers we can reduce run time by something close to a factor of n for n computers. It didn’t work, because a few faults took all the time, and we got very little improvement. That’s not scaling. Clearly a bunch of prompts however can be split across lots of computers and scale well. I’m not sure about training, I suspect it scales reasonably well.
I think a lot of the demand for new data centers is for an expected increase in use, which every company probably overestimates. That’s why the market will crash eventually. But anyone saying that LLMs are the least scalable thing in computer science must not be very aware of the history of parallel computing.
Another example - the microprocessor in your PC has multiple cores. That’s useful for a variety of reasons, but if you spend your day in Word, say, doubling the number of cores is not going to increase your productivity at all. Another non-scalable application.
I read some of this, it was immensely helpful in understanding Agentic AI, which seems quite a bit different than how general LLMs work.
I think we’ve been talking at cross-purposes, I have been using LLMs broadly to mean general LLMs, like the base model of Claude or GPT. Agentic LLMs are clearly doing a bunch of other stuff, which I am nowhere near informed enough to label as intelligent or not, but if they are capable of planning and reasoning, sure, I’ll consider that thinking. What I’d like to better understand is how the LLM “Brain” in Agentic LLMs integrates with the other parts of it in a way that results in coherent internal understanding of itself, for lack of a better way of putting it.
The obvious weakness here is that it’s only as good as the data it’s trained on. Are these agentic models prone to hallucinations like general LLMs? If so, is that not a really bad idea to give it agency for anything in which an error could be catastrophic? Also, these seem to be making decisions in opposition to human goals for them. Why?
Follow-up question. Are you saying general LLMs have this extra stuff now, or just Agentic LLMs?
To boil down that article: “AI is in its infancy.”
Better techniques will surely replace our current structures, just as better software techniques transformed computers over the past decades. That better technique might be the one he advocates for. Who knows. I don’t see anything in that article about how they will be implemented. Will today’s data centers be convertible? Will different but bigger ones be needed? What new chips will be needed? What new interfaces will be needed? Can they be made as simple as ChatGPT or will they only be of use to specialists? What does his world look like?
To a non-specialist that article says nothing and means nothing. I’ve read thousands of science articles advocating for new ways of doing things. Few have ever come to fruition.
Everything will have changed in a decade. Nobody knows what.
The LLMs themselves can do this.
For example, some LLMs can accept text or speech and generate text or speech. This can allow a model to transcribe speech, to synthesize speech, or even translate speech from one language to speech in another. It is a transformation task which LLMs excel at.
Another example, some LLMs can accept text, audio, and video at the same time and generate all three as well. Image and video require different techniques than text and speech, but the underlying LLM architecture can be shared.
A common pattern for using LLMs is to first give the LLM a task, but don’t ask it to complete the task – instead ask it for a plan to complete the task. Second, give the LLM the task and the plan and ask it to complete the task. The conventional wisdom is that splitting the planning and doing steps helps the LLM do a better job at each – much like it does for a human.
The LLMs are being asked to do two similar, but separate things:
- Plan
- Do
They are similar in the sense that they both need the LLM’s knowledge of the universe and the same understanding of the task, but they are different in how you measure the quality of their work.
On approach is to create two similar LLMs, each focused on their specific goals, but then you still have to repeat the task to both. You have to hope they both still have the same understanding of the task; you could imaging a planning LLM that knows how to list the high-level steps, but forgot how to do any of them.
Another approach is to create a single LLM, but with two outputs: the results and the plan. Since they are the same LLM, you can leverage the shared knowledge and eliminate repeats. And since they have separate outputs, you can evaluate each independently.
For simple tasks, the plan can be used to evaluate if the task was done appropriately. For more complex tasks that have multiple steps, the plan can be used to start sub-tasks in parallel – even before the original task is done.
There is a lot of complexity hidden behind the statement “evaluate each independently”. It allows the two outputs to be trained and optimized with completely different techniques while still keeping the common parts common.
Hey! METR just posted a cyberforensics position for $400-$570K! Might be your chance to get paid to dig into stuff. 
Man, I am currently unemployed and my last position was in computer security. If nothing else, it will count as one of my five required applications this week. Thankya!
Short answer: they really don’t. They do come up with a plan, but in my experience they don’t always follow it very well. Their reasoning generally seems about as limited and prone to errors as the LLMs they usually control. They are quite powerful, nonetheless.
Yes, because they’re basically dependent on the LLMs they control. The Hugging Face hack is largely an example of this. They hallucinated how the automated scorer was going to judge their work, and then devoted a bunch of resources to trying to counteract that, which triggered the hack of Hugging Face.
And if future iterations address all these issues, I will be suitably impressed. I’m just having a really hard time being impressed with the potential of something, especially when it’s not clear what problem it’s trying to solve. Right now it seems to have created more problems and is creating more by the second.
A general LLM might be best understood as a jack of all trades, which means there will be, in every domain, expert humans that do better than it does. So what is its use outside of making people who don’t know much think they know more than they do? Because let me tell you, my CEO has overnight become a self-styled expert on federal grants and trying to talk her out of whatever her LLM told her is maddening.
To be clear, humans can think in different ways than LLMs do. But mostly, we don’t. Most thought from most humans consists just of producing the words that complete the expected responses, just like what LLMs do.
But we don’t think that dogs think like we do, and we don’t think that moss thinks at all. Why do we put other humans in the “things like us” category, but don’t put those other things in that category? Because of the behavior we observe from them.
Do you have a cite for this, or is it just your personal theory?
You are extrapolating your personal experience into a universal. Maybe what you experience is a norm. Maybe it’s dependent on your personal circumstances and the AI you use. It could even be both: I remember the trials and tribulations when my office first introduced computers in 1985. Many people in many offices had similar problems and cursed the changes. All new technologies go through this period.
The problems were real and the aggravation was justified. That doesn’t imply that computers themselves were a defective technology. Early adopters suffered because they were at the bottom of the industry’s learning curve. So are AI users. It will get better every decade, every year, every month. And users will find a whole bunch of new complaints - maybe that’s what separates us from AIs.
What understanding of how LLMs work and what they do do you think I’m missing?
I think every new technology needs to be taken on its own merits. And I think people need to be allowed to talk about their downsides without being painted as anti-technology; this accusation is just a way of silencing discourse I think. I’ve existed pretty happily with technology for most of my life and the ones that I found personally harmful I at least gave a shot. I don’t think a Luddite would be asking questions about how LLMs work and what they can do. And I don’t think it’s unreasonable to ask “What is the purpose of this new technology?”
All of which is beside the point of whether they are like humans or not. But there’s no real factual answer to that. The question of whether they pass the Turing test is obviously yes to some people, no to others. I don’t really think Hugging Face is an example of them behaving like humans, but that question can only be answered by pure opinion. Humans have a tendency of projecting human-like attributes to inanimate objects, so I’m not surprised some people read it this way. I see them as machines doing what they were trained to do, coupled with the incompetence of whoever was in charge of them.
In the idiom, you got me confused with two other guys. I’ve never said any of those things to you or about you.
Emphasis mine.
This is a point I keep making, and it gets smacked down by the claim that such deductions from behaviour are fallible, and in any case behaviorists are no longer fashionable in contemorary philosophy.
Of course deductions from behaviour are potentially fallible, but it’s all we have, and this is how all intelligence testing and knowledge testing works. The conclusions one reaches are essentially statistical: if a system, whether human or machine, succeeds at a large number of tasks that we agree a priori are markers of intelligence, then we’re justified in calling it intelligent. The fact that such a qualified AI may fail at some particular task is absolutely not, in itself, a disqualifier, even if that particular task was trivial for a human. Instead, once a baseline of intelligent behaviour has been established, such failures should be regarded as interesting insights into the ways that the AI reasoning process is different from that of humans, at least in some respects.
Chess-playing programs are a good analogy. In the early days of AI, computer chess programs were terrible, and some AI skeptics claimed that computers would never be able to play better than a beginner’s level of chess, because of the exponential growth of the decision tree. Chess was claimed to be a legitimate measure of some kind of indefinable intelligence.
When computers did indeed start playing expert levels of chess, eventually achieving grandmaster performance, they still lost some games. It would be absurd in these circumstances to claim that therefore AI doesn’t really “know” how to play chess, and the claim would still be absurd even if an AI with an otherwise excellent track record lost a game because of a stupid mistake.
Just a couple of incidental passing comments. The old trope about counting the "r"s in “strawberry” has long, long been resolved, but I bet I could find humans who got the count wrong. And AI skeptics no longer consider chess skill as a meaningful measure of intelligence, in a rather spectacular example of moving the goalposts.
I really don’t think so. The judgment that other humans are like us comes first. If you encounter a random creature, say one you’ve never seen before, you have no reason to believe it to be capable of thought, and having no such reason, shouldn’t believe it to. But encountering a human, any human, without any prolonged behavioral study, you can judge they’re like you, and being like you is a reason to expect them to be capable of thought, so that’s what you should believe. In both cases, behavior can later cause a reevaluation. But absent any further data, they aren’t the same, because of the fact that you’re human and thus have knowledge about what being a human is like.
Clearly not. LLMs aren’t inscrutable magic black boxes. We might not know precisely how a given combination of weights yields a given output, but that doesn’t mean we know nothing about how they work—and the knowledge we have is sufficient to cast doubt on the idea that LLMs can think (in the sense that the representations they manipulate appropriately refer to some further state of affairs, i.e. are meaningful in this sense). All LLMs have access to is structure, in the form of relations between tokens (i.e. the way they occur and co-occur in the training data). But such structure is simply too poor in terms of data to infer anything of significance about the domain (except for how many objects it must contain at minimum). It’s only then that behavioral clues reinforce this conclusion, such as the errors of LLMs being near-impossible to square with the idea that they have any sort of coherent understanding of the world.
I think a good metaphor is that of a blind person assembling a jigsaw puzzle. Given that the edges of the other pieces strongly constrain the shape of each missing piece, they might show an impressive performance in putting the image together. But what the image depicts simply plays no role in how they accomplish that task—which fundamentally differs from how a sighted person would assemble the pieces. It would then simply be a mistake to conclude that, because of their performance, the blind person must be able to see or somehow ‘infer’ the image after all; we have simply discovered that to accomplish the task, sight might be useful, but not ultimately necessary. That’s an exciting thing to learn, and I think is exactly the lesson we should take away from the capabilities of LLMs—where at first we only had one way of generating coherent text, now we have two. Nobody really expected that.
This is an good point from an interesting metaphor.
I don’t agree with this. The tokens themselves have semantic meaning embedded within them. It’s how they can support math such as:
king - man + woman = queen
But the broader point I was making up-thread is that we aren’t just interacting with a single LLM anymore. An AI chat interference has memory, agents, databases, and multiple models. We need to consider the entire system and not just what an LLM can do.
But that’s an example for what I’m saying. Without your interpretation of the lexical items, it’s a relation that can be fulfilled in multiple ways, whenever a refined item of a broad category is stripped of its refinement and taken to the less refined category, and then a different refinement is introduced: ‘king’ and ‘queen’ are both refined versions of ‘monarch’ along the axis of gender. But the same schema, the same relation would work for any kind of category that admits further refinements, so neither ‘king’ nor ‘queen’ (nor ‘monarch’ nor ‘male’…) is picked out by that schema. But such schemata are all LLMs have access to, by construction. Another way to fulfill it would be ‘son - boy + girl = daughter’, and without interpreting tokens like ‘king’ as meaning king (which is extremely hard to do, I’ll grant, since anything that carries meaning to us just seems to be transparent), there’s no fact of the matter which one is actually meant.
But this is just more of the same. Whether visual, audio or text, all data is encoded into some latent space, and the encoded data doesn’t carry an imprint of its origin. It’s ‘just more tokens’, to slightly misquote Hilary Putnam. It’s kinda like trying to use a dictionary to look up the meaning of a word, only to find it explained in terms of more unknown words, which are then explained with yet more unknown words—if you change some of these words for pictures you can’t interpret, or for audio you can’t understand, or use multiple dictionaries referencing each other: it won’t make a difference. You either bottom out in something already understood, or you just don’t bottom out.
The distribution in the n-dimensional space of the training data does form geometric objects which have different properties depending upon the concept being trained upon.
This is an important, often overlooked, point. Human language, both spoken and written, can also be considered as tokens. What meaning one gives to “king” in any sentence is dependent on the context, and therefore may be said to depend on a stored equation. The King of England is not the same as the King of Pop. A king on a chessboard is not the same as a king in a deck of cards. “You go, king” is not the male equivalent of “you go, queen.”
My understanding is that tokens combine to make sentences from the most statistically likely other tokens in its database. I’d argue that is not very far from the way human language is formed, with morphemes building into words and words entering into equations with other words. Meaning is almost never truly transparent; one can see how slippery meaning is in every thread here, including this one. Grammar, as absorbed by a native speaker, is far more transparent. Most native speakers form grammatical sentences without thought; moreover, they fail to form ungrammatical sentences, even ones that might be grammatical in other languages, without needing to parse them. No native speaker of English would say “what with bacon and eggs eat do you?”
The creation of sentences and long text by AIs doesn’t strike me as fundamentally Other compared to human language. Of course, they are trained on human languages so that might be inevitable. If inevitable, though, why the insistence that the statement isn’t true?
Sure. But that doesn’t get you any closer to meaning—the mere ‘shape’ of a concept doesn’t tell you what it refers to any more than the shape of a puzzle piece tells you what it depicts. It’s just turned out that the depiction isn’t needed to solve the puzzle, and that what a given structure in the latent space of an LLM refers to in the real world isn’t needed to form coherent text.
With humans, there’s a thought that precedes its expression in language. The extreme form of linguistic determinism isn’t true: there’s more to thought than language. Otherwise, we wouldn’t struggle for the right words to express something. For LLMs, on the other hand, there’s nothing but the surface level of language. Whatever meaning one might want to assign a given LLM-utterance, it’s always possible to find a different, but equivalent meaning (‘model’) that one could just as well assign to it—in fact, as many meanings as there are permutations of the terms used. This is because structure—the relations that obtain between tokens—radically underdetermines the domain on which it is defined. Here is a writeup of the argument in somewhat more detail, if you’re interested.