The next page in the book of AI evolution is here, powered by GPT 3.5, and I am very, nay, extremely impressed

Cafe Society
981

No it isn’t. A computer is a digital adding machine. A brain is a self organizing analog system that processes sensed data. The brain strives to understand it’s environment. The computer tirelessly adds whatever numbers are fed into it.

The phrase is a throw away euphemism like ‘the bridge is the brain of the ship’ or ‘White Plains is the brain of IBM’. The amazing thing about GPT is what an adding machine can accomplish. To embue it with brain like characteristics is mysticism.

Oh yeah, the second thing was talking to a union organizer.

I was an Iconoclast, I wore plaid ties.

In the summer the IBM band would play during the lunch break. We’d sing along and the last song, led by Marie the bldg 2 receptionist, was Ever Onward IBM. Could have been choreographed by Meredith Wilson.

982

A Large Language Model is a self-organizing digital system that processes data. It is explicitly modeled on how the brain ‘learns’, and it appears to be doing the same thing.

Whether the data is ‘sensed’ or just loaded is completely irrelevant.

And now we have multi-modal AIs that can integrate sight, sound and text into a coherent whole:

From the paper linked in the article:

Lots of examples in the article.

983

No, running an LLM does not change the structure of the supporting computer hardware. The program is simply resident. There is nothing self organizing about it.

The real miracle here is the high level language and the software management procedures that make such an enormous program possible. The programming language is so far removed from the instruction set of the computer that the program is computer independent. The compiler and possibly some other intermediates determine what instructions get to the computer and the loader or similar utilities determine where they reside. There is no consistent brain like layout.

As your examples show, the resulting computer output can simulate human behavior. But that is all. You know it is still just an adding machine. Attempting to explain behavior as ‘thinking’ rather than the result of brilliant program management is an insult to the developers.

984

I completely disagree. The underlying computer hardware is just the substrate on which the self-organization happens. Saying it’s ‘just an adding machine’ is like saying the brain is 'just chemistry '.

Again, the internal steucture of a neural net is evolved, not designed. No one understands how they do what they do in all but the most trivial of cases. We can see information and processing structures similar to those that evolve in the human brain. The capabilities of these LLMs emerged, and was not designed or even expected in some cases.

It takes extreme effort to decode what’s going on in even a simple, single layer neural net with 50,000 parameters. And even there we see emergence of complex ‘circuits’ that solve problems. ChatGPT has 175 billion parameters arranged in a 96 layer deep learning network. GPT-4 is rumored to have 100 trillion parameters, roughly matching the number of connections in the human brain.

This is not simple ‘adding’ any more than your thinking is ‘just chemical reactions and electrical impulses’. This may describe the mechanism way down at the lowest levels, but is pretty much irrelevant when trying to understand the high level organization and operation of the logical structure on top of it that actually makes up a brain - or a computer neural net.

985

The neural net is part of the software not the computer. The computer is the only active component in the system and the computer can only do 5 operations: ADD, AND, OR, XOR, NOP. It is an adding machine, no more, no less. If you have evidence to the contrary please show it. Check out the operations in the high level language used.

The neural net and the LLM process is not witchery. It is logistics and engineering. It is the product of brilliant minds and hard work. Some of the output is surprising, but it is the result of a well understood, managed process. It may perform beyond expectations, but it does not perform outside of it’s design properties. There is no emergent magic required to explain the output surprises.

This doesn’t detract from the amazing accomplishments of computer software. It just places boundaries on how we interpret what we observe. The neural net isn’t a thing, it’s just a series of equations in the line of execution. It all goes through the binary adder. As for the output, what you see is what you get. The output of an LLM is just that, the output of an LLM. It doesn’t understand and it doesn’t communicate, it just moves data. Anything beyond that comes from the observer.

My question is, if a computer is more than an adding machine, what is that ‘more’, where does it reside?

Thanks for the interesting discussion.

Crane

986

This is just a bizarre form of reductionism. Everything in physics can be simulated by a sufficiently powerful “adding machine,” so at worst, you can create an exact replica of a human brain just by simulating the behavior of the atoms.

Ask the same question about humans and you’ll have your answer.

987

I believe you. There was a period of time when major computer vendors were trying to calm public perceptions of “the great takeover by intelligent machines”. When Digital Equipment Corp introduced the first model of its VAX line of computers – more powerful mainframe-like successors to the PDP-11 minicomputer – they specifically built the cabinets to be slightly lower than average human height to avoid the impression of the towering overlord. And I always thought – admittedly without real evidence – that the very prominent red “emergency off” button on IBM’s line of System/360 and System/370 consoles was as much a PR stunt as anything of practical value. Kind of ironic that IBM Research then invested considerable sums in a system that could beat the best humans at the intellectual game of Jeopardy!

I have to wonder how the current state of DeepQA compares with GPT 3.5. Advancements in GPT have been so rapid that, along with IBM losing confidence in the commercial potential of DeepQA, I think GPT is probably in the lead. It may not do as well as Watson did even back in the day, but only because it hasn’t been specifically trained in that domain. It potentially could be even better, but ChatGPT suffers from those annoying “AI hallucinations” whereas Watson was quite good at confidence-rating its proposed responses.

988

“A neuron in the brain is a chunk of meat that simply fires an electrical impulse when certain conditions are met, No more, no less.”

When analyzing complex systems, the lowest levels of operation tell you almost nothing about the system itself and its capability. But we know of no reason why we can’t digitally simulate the behaviour of a neuron or a network of them. And when we do, the network exhibits the same kind of behaviour that neural networks in animals do. If you think there is a fundamental difference that makes software-based neurons fundamentally incapable of working like biological ones in ways that matter, it’s on you to back that up. All the evidence we have so far suggests otherwise.

The process for training them is understood, The process by which they develop answers is not. Not even close. We’re at about the level of understanding of the higher level complexity in LLMs and LIMS as we are in understanding human thought by studying the nervous system of flatworms.

This is just wrong. I have already linked to several papers describing emergence in Large Language Models. They have exhibited all kinds of abilities that absolutely surprised the people who built and trained them. And these abilities ARE emergent. They just show up at some combination of parameter size and training quantity. They are unpredictable, and show up in different models at different times due to subtle differences we don’t understand.

We have no idea what will emerge next as we scale up. For that matter, we don’t even know what capabilities have emerged now, as there may be some that don’t map to human activities like translation or writing poetry, so we don’t know how to prompt the thing in a way that illuminates them.

I was just listening to an interview with a scientist at OpenAI. He was asked what the biggest misconception was when lay people try to understand these models. He said that it was “that all they are doing is statistical word generation with no real understanding of what they are outputting.”

He pointed out that the statistical word thing is part of the training process - you get the AI to give its list of predicted words and compare it to the actual probability distribution, and if it’s wrong you feed the errors back into the net using back propagation. It uses gradient descent to determine which parameters to change, and by how much. Repeat billions of times until its accuracy is high enough.

Those parameters that are changing are like synapses in a brain. They represent the strength of the connections between nodes in the network. By modifying the parameters new connections are formed, old ones removed, and the weight of each connection affects how much influence they have. After iterating trillions of times, the result is a fantasitically complex network of 175 billion unique connections that we don’t understand.

So when the trained AI is giving its answers, we don’t know how it’s doing it. Yes, at some point a transformer gets a list of token probabilities, but how those probabilities are determined inside the network is unknown, and according to that researcher there are lots of signs that real ‘understanding’ is going on. It’s at best an open question.

The nature of complex systems is that because of emergence, they are greater than the sum of their parts. The ‘more than just adding’ part emerged through the development of a giant neural network.

For example, for a long time ChatGPT basically spit out noise when asked to translate between languages, but at a certain number of billions of parameters, suddenly it could translate.

ChatGPT could not do two-digit arithmetic for a long time, but suddenly it could. Examination of the mechanism of its ‘brain’ that does that showed an unexpected phase transition (emergence) from rote memorization/statistical word lookup to an evolved ability to do general two digit math through trig identities and fast-Fourier transforms, which it apparently developed on its own. No human wrote an FFT algorithm or a trig library for the LLM. It built its own. And no one knew it was capable of it.

Let me give you a very simple thought experiment: You can write very simple software to iterate finite state automata. Less than 10 lines of code. But there are some rules that, when iterated, produce very complex unexpected output. Rule 30 can generate the patterns we see on conch shells and elsewhere in nature, and exhibits true chaos and can be used as a random number generator (Wolfram Alpha uses it that way). Rule 110 is Turing complete, and can be used as a universal computer. So where is all that complexity coming from?

In an LLM we are iterating over a much more complex information space, trillions of times. And unlike the cellular autonoma, a neural net is adaptive and modifies itself on each iteration. The level of complexity coming out of this process is immense.

989

What is the commercial application of these systems? It’s hard to recover a billion bucks.

990

I haven’t been following it very closely, but initially I believe IBM saw potential in DeepQA as a medical advisor. This was apparently a significant failure, so they sought applications in the business world, such as UI testing. No idea where that went. But I can see potential applications in the area of advanced semantic search.

991

Quite the contrary, to claim that something that performs the same tasks as a brain must nonetheless lack some sort of essential brain-ness is mysticism.

Are you also one of the folks who insists that there’s no such thing as “memory” in a computer, just “data storage systems”, because “memory” is a thing that only souls brains have?

992

This is precisely the basis of my argument with @Half_Man_Half_Wit. Of course we’re some distance away from being able to declare that any AI actually “performs the same tasks”, but we seem to be getting closer, not just in a timeframe of years, but in significant advancements in testable intelligence in the latest neural nets that occur in a matter of months.

993

There are so many applications for this tech that I could probably type for an hour just listing all that I can think of.

The DeepQA stuff is perfect for things like Asset Performance Management, in which large amounts of performance data are used to predict failures of machines or to capture lost efficiencies. The APM market alone is about $8 billion per year.

Operational performance is another one. Both managerial and technical. OEE and other tools for mathematically analyzing operations will be revolutionized. OEE software is about $2 billion per year, and that’s just a fraction of what companies spend on software, engineering and analysis of data.

Those are just two from the field I am most familar with. Anywhere that large datasets need to be analyzed will be revolutionized by this software.

As for ChatGPT and its future ilk, a good rule of thumb would be that if you are a white collar worker who does not have valuable domain-specific skills in manipulating the real world, your job is at risk. Again, there are so many applications it’s hard to know where to start. But let’s follow a potential employee through this new world of AI:

Employee needs a job. He gets his AI assistant to write up his resume. It is then submitted for him to every employer looking for employees who match that resume and your job requirements.

Your resume’ arrives at the employer. An AI ‘reads’ it and performs the initial choice of rejecting, phone interview, or in-person interview. If it’s a phone interview, you will be talking to an AI. Rejections come back almost instantly, with an AI helpfully explaining why you weren’t accepted. Fast turnaround means you can update your skills faster, apply for more jobs, change your resume, etc.

You get the job offer. You show up on your first day, and a company AI fine-tuned on all the HR manuals and other relevant corporate info gives you your welcoming package, tailored to your job description. you are assigned your own AI instance, which will act as your ‘buddy’ in showing you the ropes. Any questions you have get answered immediately.

You go to your first meeting. After the meeting, by the time you get back to your desk you have a transcript of the meeting, bullet points for any action items you are responsible for, and summaries of all pertinent information. This happens after every meeting.

A couple of days later, the AI notices that you had an action item for completing some code, but you haven’t checked anything in. It reminds you of your promise, asks you if you need help, does a code review for you, and gets you across the finish line in time.

Of course while you are coding an AI is helping you all the way (see: Github co-pilot).

You have to produce a progress report. You talk it through with the AI, which has all the stats for where your team is compared to where they should be. It produces the report for you, with beautiful graphs and tables. What would have taken you a half day or more was done in a couple of minutes.

Your project lead, in the meantime, has his AI producing Gantt charts, warnings about the critical path, projections for completion, which team members are falling behind, whatever. Project management is now 80% AI.

I could go on. How much is all this worth? Well, OpenAI Foundry is selling corporate plans. A ChatGPT corporate plan is $250K per year. They have a new model that’s much more advanced, though. That one is going to sell for $1.5 million per year. My prediction is that almost every large corporation will either buy this or a competitor’s equivalent. Even 1.5 million per year represents only 5-10 emp;oyees. They’ll probably save more than that just on HR manpower, let alone all the productivity enhancements the AI will bring to every employee.

This is an excellent substack that covers various AI issus. Here he’s talking about pricing of LLMs:

OpenAI's Foundry leaked pricing says a lot – if you know how to read it

994

For kicks, I tried asking it to explain some code that I just wrote from scratch. It did an excellent job, only making a slight error in the motivation for unrolling a loop. It even correctly inferred the behavior of some macros that I hadn’t actually provided (some assertions and bitwise ops). On one part, it repeated a comment that was already there, but gave additional background to the motivation.

It did a less-good job at a code review, suggesting some things that were already in there. But nothing worse than what I’ve seen humans suggest.

Maybe I’m just biased since it complimented me with “Overall, the code looks well-written and efficient” :slight_smile: .

I don’t think we’re that far from the point where AI code reviews might be acceptable. Especially if the AI can be pre-trained on the whole codebase.

995

Yeah, At least at the beginning I suspect that we’ll only trust the AIs for initial code reviews to check if the code simply meets thr corporate coding standards. Full code reviews will be done by people in the near future, at least for all mission-critical code.

But as time goes on and we figure out the accuracy of these AI code reviews and thr AIs gain more feedback from the results of those reviews, they’ll eventually get good enough to be relied on. i could see something like an AI review being part of thr checkin process. Try to push your code to the repository, and first the AI will review it and refuse to do it if the code is flawed.

The next few years are going to be wild.

996

I think we’ll see an intermediate stage where junior engineers are highly encouraged to get a preliminary AI code review, which only gets passed on to the senior devs once it passes. I agree that it’ll probably be a while before human reviews are eliminated entirely, though (where “a while” may only be a few years).

997

Yep. Agreed.

998

Supervenience is the relationship between physical and mental properties. Consciousness emerged from and supervenes on the functional state of a physical brain. The physical brain is complex, but other than that, it’s nothing special (call it a super-complex adding machine if you want). Re-create that brain organically and you re-create the consciousness. Why could you not re-create that brain inorganically and get the same result?

I see no reason that consciousness could or would not supervene and emerge from inorganic hardware as cognitively complex as a conscious organic brain (and remember, many animals are conscious, not just humans and a handful of higher mammals…like cats :smiley_cat:). In fact, you can strip away much of a conscious brain (those parts not involved in higher cognition, and even some parts that are) and it will still remain conscious (studies confirm this. Google cites yourself if you want…I’m going to bed).

There are, of course, different types and degrees of consciousness. Being conscious simply means that you are aware. Being self-conscious means you are aware of being aware. I believe consciousness is a lower-order mental state, while being self-aware is a higher order (though it’s more complicated than that, with steps in between). If consciousness supervenes and emerges from physical brain matter, then perhaps self-consciousness supervenes and emerges from the basic state of consciousness.

Using that simplistic/dumbed-down criteria, I believe we are on the verge of creating conscious AI (whether we know it or not). And at the rate we’re going, I believe it’s going to happen relatively soon. If, and when that occurs, self-awareness will follow, though it may take much longer to emerge. Or, maybe not.

At the end of the day, our current understanding of consciousness is rudimentary at best. There’s a lot of psychobabble being flung to and fro even between credentialed scientists on the matter. Maybe there does need to be an organic substrate from which consciousness needs to emerge (and that can be created in bio-labs). Or, maybe God, or the Flying Spaghetti Monster needs to put it there through Intelligent Design :smile:. But, IMHO, consciousness can be organic, or inorganic, and it doesn’t involve magic, and we’re going to create it wittingly or unwittingly very soon. That need not be a bad thing. Just accept it and go with the flow. You and I are just cogs in something much bigger.

999

I’d have to look it up, but some decent % of the lines of code at Microsoft are written by github copilot. Reduces overall coding time by like 10%. I wouldn’t be surprised if eventually the big tech companies finetune some massive models on their internal git repos and new hires are basically relegated to tweaking model outputs and learning to become LLM gurus.

1000

Nope, memory is just a label.

Help me out here, I don’t understand what you find brain like about a central processing unit?