The Miselucidation of Whack-a-Mole

The BBQ Pit
61

Without getting into another extensive back-and-forth about what now seem to be largely differences of opinion about the future of AI, I just want to highlight this part where I believe you’re misrepresenting my statements, essentially doing what you’re accusing me of. As you say, this is indeed the Pit, so if I wanted to insult you, or to impugn your character as you suggest, I wouldn’t need to be coy about it. I have no wish to do so and I would think that would be perfectly clear.

I didn’t equate you with the misguided skepticism of Dreyfus or Searle; what I was trying to say was that some of your dismissive statements “effectively” align with their skepticism; I’m thinking of things like this, which followed on to your dismissal of GPT as nothing more than a stochastic parrot:

There are two questions here, so let me deal with the first one first. If you’re in any way suggesting any sort of comparison between Eliza and the latest generation of LLMs, that’s just ridiculous. They’re not remotely comparable. It’s like comparing a 1930s era hand-cranked calculator with a modern supercomputer that can play grandmaster level chess or score better than the vast majority of humans on intelligence tests, on the basis that both machines do computations so they must be qualitatively the same. That the transformer model provides a novel approach to neural networks is largely just a matter of efficiency, allowing sequences of tokens to be processed in parallel rather than sequentially, and is only one of a great many factors that make LLMs so powerful, not the least of which is the massive computer power behind them.

I played with Eliza a bit when it was first released to the public in the late 60s, and I have it still (only due to its interest as a historical artifact) on the SimH PDP-10 simulator on my PC. In fact I just fired up my virtual PDP-10 and ran the original Eliza program and posed the same question to it that I had earlier posed to ChatGPT 3.5. Here are the two conversations.

FYI, in the Eliza dialog, “WORKER” is the name of the LISP function that starts the program.

Eliza:

.r eliza

(WORKER)
HELLO.

HOW DO YOU DO. PLEASE STATE YOUR PROBLEM

IS LANGUAGE FUNDAMENTAL TO HUMAN COGNITION?

I AM NOT SURE I UNDERSTAND YOU FULLY

IT’S A PRETTY STRAIGHTFORWARD QUESTION

PLEASE GO ON

IS LANGUAGE INTRINSIC TO INTELLIGENCE?

WHAT DOES THAT SUGGEST TO YOU

IT SUGGEST TO ME THAT YOU’RE AN IDIOT.

WHAT MAKES YOU THINK I AM AN IDIOT

ChatGPT:

Is language fundamental to human cognition?

Language is often considered fundamental to human cognition, and it plays a crucial role in human thought and communication. While it’s not the only cognitive tool humans possess, it is one of the most distinctive and powerful ones. Here are some reasons why language is viewed as fundamental to human cognition:

  1. Complex Communication: Language allows humans to convey and share information with great precision and complexity. It enables us to express abstract ideas, discuss the past and future, and convey emotions and desires. This level of communication is unparalleled in the animal kingdom.

  2. Thinking and Problem-Solving: Language is closely tied to thinking and problem-solving. It allows us to formulate and manipulate abstract concepts, plan for the future, and engage in logical reasoning. Without language, many complex cognitive tasks would be severely limited.

  3. Social Interaction: Language is a key tool for social interaction. It enables us to establish and maintain relationships, cooperate with others, and transmit cultural knowledge across generations. It plays a significant role in human culture and society.

  4. Learning and Education: Language is essential for learning. It is the medium through which we acquire knowledge, transmit information, and educate future generations. Without language, the process of teaching and learning would be drastically different.

  5. Memory and Thought Organization: Language aids memory and thought organization. We use language to label and categorize our experiences, making it easier to remember and make sense of the world around us. It helps us create mental models and narratives.

  6. Self-Awareness: Language contributes to self-awareness and introspection. It allows us to articulate our thoughts and feelings, reflecting on our own experiences and motivations. This capacity for introspection is closely tied to our ability to use language.

While language is fundamental to human cognition, it’s important to note that it is not the sole cognitive tool humans possess. Humans also engage in non-linguistic forms of cognition, such as visual and spatial reasoning, mathematical thinking, and emotional processing. Additionally, there are individuals with language impairments who still demonstrate sophisticated cognitive abilities, suggesting that other cognitive mechanisms can compensate to some extent for the absence or limitations of language. Nevertheless, language remains a central and distinctive feature of human cognition.

Moving on to your question about why I’m impressed by GPT, the above example is a small part of the answer, but I want to stress that I am not – as some might suggest – simply bamboozled by the authoritative-sounding language. Yes, GPTs can be hilariously wrong – sometimes about apparently simple matters – but they are right much more often, as evidenced by the extensive list of advanced tests for knowledge, intelligence, and problem-solving skills that they have aced. I listed some of those tests above (and it’s only a partial list). It’s not anything in particular about the architecture per se that impresses me, it’s the end result.

Yet another skill that I found impressive was in connection with a question designed to test a person’s financial numeracy that was discussed in another thread. The question itself was relatively simple: given a table of credit cards organized into two categories based on whether the cardholder pays off balances in full each month or carries a balance, describe the differences between these two categories. The impressive part was that there was no easy way to convert the table into a well-formatted question for ChatGPT; the table was an unholy mess, with everything misaligned. A human would have had a hard time with it. ChatGPT interpreted it perfectly, demonstrating a powerful utility in processing unstructured data, something that computers traditionally struggle with.

Yet another example is its ability to do language translation with an apparently deep understanding of real-world phenomena and nuanced meanings in both the source and target languages. In the early days of AI, researchers despaired about how to convey contextual understanding to machine translation; LLMs appear to do it effortlessly.

62

One cannot argue with results. (Or can you? I do not even consider human cognition as particularly impressive (relatively speaking!) or a desirable baseline to imitate in the long term if you want artificial intelligence. And, separately, my point was that LLM performance may be stunningly impressive compared to ELIZA, and even on its own impressive and even useful, provided one understands more or less what it is doing, but, no, it is not intelligent in the sense we are discussing in this thread.) But I think that if you want, for instance, better semantics, let alone critical faculties then you have to at least accept the possibility that the right way to proceed is not merely to train the exact same GPT model except using more memory, that you need to modify the architecture radically. (Machine learning can help you do this!) GPT is not the Holy Grail even of LLMs; if you care about the end result then the architecture does matter (since that is what drives the computation given a quota of of time and space resources) but you should not regard GPT as some kind of ultimate stage and therefore ascribe “intelligence” to it willy-nilly, even if one can go into the network and pick out neurons that “understand”, e.g., animal names.

63

I don’t want to equate you with the most evil person in history, but I do need to point out that your love of dogs effective aligns you with Hitler.

I mean, if you aren’t trying to suggest SoaT is like these other people, then why did you compare them? You obviously did want to equate his statements with theirs. Give a mea culpa and move on.

64

True, though it’s the only example we have so far?

I always liked Fred Hoyle’s “Black Cloud” for an example of a larger-scale intelligence.
A fun book; the tech is dated but the science is mostly right…

65

As it stands, GPT is a complement to human intelligence, not a substitute. It’s generally understood that computers will someday surpass human intelligence first narrowly (accomplished decades ago) then broadly. So cheerleading is besides the point.

Can it be stopped? Not really, but it can be regulated. Just like cars or any other consumer good or industrial process. Humans rely heavily on stories and labels. So if humans want to use our AI friends productively we require a short insulting term to highlight their shortcomings. Now you could say that they are like a newbie reporter that doesn’t know how to fact check and absolutely requires a knowledgeable editor who carefully reads their output sentence by sentence with a sharp red pencil. That isn’t awful in terms of accuracy, but the problem doesn’t really require accuracy. We need something brief and memorable, something that instantly emphasizes AI’s shortcomings. In short, we need a caricature, not a description. Stochastic parrot leads the race so far, because rule of funny.

66

“Most of the damage that the computer could potentially cause depends less on what the computer actually can or cannot do than on the qualities that the public attributes to the computer.”
– Joseph Weizenbaum, author of Eliza, in Die ZEIT, 03/1972, p. 43

The quote is outdated and I’d say, “User” rather than “Public”, but at least attribution is something under our control.

67

GPT doesn’t have to smarter than humans: being faster is already a superpower, for both good and bad. Being able to do things at huge scale that humans can only domat smaller scales or with great effort is very powerful. Sometimes quantity brings new qualities.

For an example of a bad power, it used to be that most people were relatively safe from having their location data abused, simply because there was so much of it. If you didn’t give the authorities some reason to look into you, you were safe.

But with AI in the mix, your location data is easily queried. Apply for a job, and an employer might ask a service to do an AI check, and it might return how often you were late to your previous job, whether you hang out in bars, whether you leave work earlier or later than other employees did, yada yada.

On the good side, consider the amount of data now being received by the new generations of telescopes, Terabytes per day. Humans can’t hope to manually examine it, so it’s going to be a platground for AI. We have already had an AI find a few candidate alien signals in old data that were missed by humans the first time around.

Big data applications will all be heavily AI based soon.

No, it probably can’t. Unless you can regulate China and Russia and Iran and everyone else going full speed on AI. All you can do is hobble the nations that allow themselves to be regulated. Also, the NSA and FBI and such aren’t going to slow down either.

Certainly government could control the large cloud providers and the huge public-facing LLMs. But AI development in the open source community is already threatening the large companies and GPT 3.5 level AIs are now running on consumer grade hardware.

Regulating AI will just drive development underground, give other nations an advantage, and inhibit our ability to develop AIs to protect us from malicious AIs. It will also give the public a false sense of security.

Like it or not, the barn door is fully open. The ship has sailed. AI is moving too fast and is too valuable to stop. All you can do is hurt yourself trying.

Note I’m talking about regulatingbthe tech itself. There is some room for legislating rules around IP use and such, so long as they aren’t too stupid.

68

Substitute “Computers” for “GPT”, and this was true 50 years ago. Agree with your first section though.

[I knew that would come up: Ha!]
No it can be regulated, because we have a head start. Training these models requires enormous amounts of manpower, which we mostly farm out to English speaking countries like India and Kenya. China has plenty of cheap labor and while I don’t doubt they will want to keep up with the West, I suspect Chairman Xi will find himself uncomfortable if ChatCPC starts elucidating on the Chinese Communist Party and Pooh bears.

Luckily regulation can’t be stopped either because of product liability. Which is a very good thing, because regulation promotes growth and human progress. People don’t want to buy cars if they think that they are a rear-end away from a conflagration and people don’t want to fly planes if they see plane crashes on television every 8 weeks. (That was a real concern during the 1970s, given projections of future air travel. The response was to double down on safety, an effort involving government, airlines, aircraft manufacturing, and the professionalism of the pilot community. Luckily practical people were in charge, mostly, so aircraft safety expanded faster than air travel or glibertarian policy making.)

I say that AI poses the greatest risk conceivable to human kind, because it can be used for weapons development, just as technology in general can. There are other risks which we have less experience with outside of fiction (and boy do we have fiction) but I suspect it will be the familar ones that kill us. But who knows?

LLMs are pretty exciting and pretty insightful. That they can simulate authoritative speech shines a bright light on human limitations. I thought I had a modicum of free will, but on reflection I just rehash what others have written with a little tweaking here and there. It’s chatterbots all the way down.

69

Trining them no longer requires that kind of manpower. AIs can train AIs. In fact, this seems to be an incredibly powerful technique, producing small AIs that can run on consumer gear that are as smart as GPT 3.5

Also, there are plenty of pre-trained models out there which you can take and use as-is, or extend with more training or fine-tuning. Training a 7B or 13B parameter model is a lot less energy and compute intensive than training the 170B parameter GPT 3.5 or the trillion+ parameter GPT 4V.

Llama, an open-source GPT 3.5 level model, was trained on 1.4 trillion tokens for $1 million. That’s small money to any reasonably sized tech firm.

Bear in mind that this is all done using generic GPUs. Chip manufacturers are now working on dedicated chips to accelerate the typical bottlenecks in LLMs. Expect to see Moore’s law continue to work here for a few generations. Eventually, training will be something that anyone could do with an OpenCrawl dataset.

In my opinion, AI could well become dangerous in many areas. But I also think that we will be under attack from AIs spawned out of the usual places - Russia, China, Iran, etc. The places that launch the most Cyberattacks today. Our best defense against that will be to maintain our lead in AI and go full steam ahead, because we will need AIs to protect us from other AIs. Regulating ourselves while our enemies go full steam ahead is a really bad idea.

Also, I really hate the idea of government having a monopoly on the best AI tech, which I think would be a likely result of heavy regulation. It will be used against the citizenry for sure. It probably already is. The best defense is to allow everyone access to AI tech.

70

Ok Sam, but there’s a contradiction in your text. It’s a given that Russia-China-Iran can work with Llama. The extent to which they can push the AI frontier though is by no means clear, both because of their inferior technological capacity (for now) and the unwillingness of authoritarian governments to advance difficult to predict speech.

And there’s a huge assumption that regulation means, “Regulating ourselves to death.” This hasn’t happened in car safety and it hasn’t happened in aircraft safety, though it could be argued that both are somewhat overdone. Could be. Progress hasn’t stopped in either industry though - that’s not the way things work in modern democracy. And it’s not like there’s a shortage of funding for AI research. There’s a reasonable debate to be had about the scope and intensity of regulation. An all encompassing skepticism of it though is ahistorical and unhelpful.

71

Completely agree with the first part, and in fact I was going to say something to that effect but by the time I got to the end of my long post I had forgotten. GPT (and LLMs in general) is a breakthrough technology, but certainly not any kind of ultimate end stage, and indeed there will never be one. But I suspect that even its designers were surprised by some of its emergent capabilities. I think that something like GPT will eventually be a component – but only one component – of a future artificial general intelligence (AGI). It will probably be most valued for its capabilities as a natural language interface. The fact that GPT does so much more than that is rather amazing, particularly since much of it is emergent. I think the reason ChatGPT and its ilk have become such big news is the combination of a breakthrough technology and the fact that it’s widely available in a form that the public can easily experiment with.

But on the subject of whether it possesses “real” intelligence or “real” understanding, I think this is a rather pointless philosophical quagmire that’s been thrashed to death here several times. From a practical perspective, the useful question we can pose is whether it appears to understand what we are asking it, and whether it responds – with a high degree of consistency – in a way that we consider to represent intelligent behaviour. If we can answer in the affirmative to both of those, the fact that it occasionally gets even simple things wrong is not evidence of anything except that its intelligence is not human intelligence, and its strengths and weaknesses are different.

I have little patience with those who cite superficial descriptions of how it works under the covers as evidence that it doesn’t actually understand anything at all. Is there any doubt that if presented with something like GPT-4 say about 20 years ago, that there would be little doubt that we were dealing with an intelligent entity?

The core of the metaphysical problem here is that AI advances in increments, LLMs being a particularly big leap, but still an increment. Each incremental advance involves some new technique or technology, and as AIs continue to do more and more things previously thought impossible, we become acclimatized to it and, bolstered by a superficial understanding of how it works, dismiss it as just a computational “trick”. As the AI pioneer Marvin Minsky once said, “when you explain, you explain away”. The plain fact of remarkable behaviours that are right in front of us suddenly becomes tainted by the perceived aura of technological chicanery.

72

My first intention was to give that whole text a wikipedia treatment, with numerous [citation needed] and [according to whom], but since I can’t seem to find a way to get blue text, I’ll skip it.

Did you post this as an example of something that impresses you? Of course it’s better than Eliza. But the text is full of weasel words, allusions and a language that is less than precise. If someone had posted that on Wikipedia, its users would be on it in minutes, tearing it apart.

I find it grating that it uses first person plural. It’s on the level of pop-sci magazines.

With time, it’ll get better of course. From this and all the other examples I’ve seen, I’m impressed, but maybe not the way you are. It’s like MS Word doing a summary of a text. Of course it’s bad, but actually not as horrible as one would expect.

73

You seem easily annoyed. New Year’s hangover, perhaps? I’m sorry (not really) that you don’t like the stylistic use of the pronoun “us”, and you’ll have to do better than “[Wikipedia] users would be on it in minutes, tearing it apart” if you expect to be taken seriously.

This is a random example of an interaction with a research release of experimental software for evaluation purposes, not a submission to Wikipedia required to provide citations, though some LLMs do provide their sources.

The more pertinent question here is what part of that is factually wrong. It’s possible that something is, though nothing is obvious to me. Maybe instead of vapid, empty snark you could enlighten us with actual factual corrections. Or just sleep off your hangover instead of bothering the rest of us.

74

I don’t use any drug stronger than coffee. So butt hurt snark fail*.

It’s quite simple, actually. Phrases such as “is often considered,” “language is viewed” and so on are inexact and serve no real purpose. They only muddy the waters. Other phrases like " Without language, many complex cognitive tasks would be severely limited." are true, for a given value of true. So is the phrase “Up to 40 % off” - i.e. you may only get 2 % off, and someone, somewhere will get 40 % off. Maybe. Semantically it’s a null phrase. Which complex tasks? How are they limited?

The whole text is vague, sloppily written and might pass muster for someone in junior high.
Again - I find that impressive. A machine that is actually answering a fairly complex question on the level of an average kid in their early teens.
But not game changing. Quite possibly it will eventually become game changing, but we’re not there now.

*ETA: Why would you go on a personal attack? This is the Pit and it was quite mild, but still. I think you are showing who is easily annoyed.

75

If we think about the puffy bureaucratese of corporate press releases, a lot of office emails, government website content, corporate annual reports, many Dope posts, etc., we see that the bloated but vague style is used a lot by a lot of humans.

The folks who churn that stuff out the old fashioned way will certainly welcome a power tool to generate it for them. Many already have.

As more and more of the content on e.g. new sites or feeds comes to be written by them, the rest of us will slowly become (more) accustomed to that sounding like good writing.

Another example of modern tools making a few people’s lives easier and everyone else’s shittier.

76

Which of course raises the question of why there is so much of that bullshit around, and whether it is actually of any use to anybody… :wink:

77

Yeah more of that than even the High School Freshman essay.

I know that these generative AI models can produce in other styles, but dang every time I read something out of it I am reading that same “voice.” The one that says in fifty words what could be said in ten.

78

To quote Heinlein: if you can’t write 10,000 words of bull on a bull subject, you may end up having to actually work for a living…

79

Those may be valid criticisms if applied to a paper submitted to Nature. They are NOT valid criticisms of an informative response intended for a general audience. In that context, your criticism is as frivolously insubstantial as your dislike of the pronoun “us”. I asked you for factual rebuttals, and I have yet to see any.

“Junior high”? Any kid who wrote that from his own general knowledge without doing any subject-specific research should be in graduate school, not junior high.

Allow me to offer a small excuse and even a mild apology for it. I’m just sick and tired of AI “skeptics” who are so adept at moving the goalposts of what actually constitutes “real intelligence” and “understanding” that they’ve put the goalposts on wheels now, and they almost never stop moving.

80

What goalposts? When have actual “AI skeptics” ever set goalposts for artificial intelligence? We barely understand what human intelligence is - how could we even start to define AI?