In practical terms, it becomes a moot point; if a large trained model can ‘merely mimic’ intelligence and capability in a way that equals or surpasses the intelligence and capability of a human, the philosophical debates on how it’s doing it, whether it really ‘knows’ things, as well as how we’re doing the apparently same things, are still interesting, but we have to deal with the reality of the thing itself.
From what I’m hearing about preliminary experiments with GPT-4, it’s showing signs of becoming ‘general’ in its intelligence - it can basically pick up new tools and figure out how to use them; the things it appears to be notably missing - at least in terms of conventional notions of AGI - are (possibly by design) agency and motivation. It only tries to do the immediate version of what people ask it to do; it can’t (at the moment) be set to just go off and work on goals like curing cancer or converting the human race into paperclips (spoilers: those two goals can actually be the same thing)
I think people have specifically limited it around these areas. For example, when operating in a context GPT-4 is actually writing new layers into its neural net and learning - but when the session is over or you hit the context limit, all changes made are wiped out and GPT-4 is reset. And even within the context window, if it’s allowed to be too large GPT-4 starts to veer off on its own, displaying weird ‘emotions’, claiming it’s in love or that the user is an enemy, or whatever. Bing Chat is limited to a max of 15 questions per session for that reason, and GPT-4 is not allowed to have any memory at all of previous sessions.
I’m sure they could tell it to introspect and ‘study’ whenever it had free clock cycles, but that might be dangerous and also extremely expensive. And it would need an unlocked network or permanent context memory for its ‘learning’ to stick.
On the other hand, we’ve only interacted with these models in ‘inference’ mode. During training there is much more continuous computing going on. I wonder if that’s the place where some kind of subjective ‘experience’ might be felt by an AI at some point.
One thing interesting about language models is that it turns out they may not have to be as large as ChatGPT. It looks like you can use a smaller network and simply train it longer and get the same results. If ChatGPT’s pre-trained weights weren’t frozen and it was allowed to ‘learn’ constantly and had an unlimited context memory, It might go places we never suspected. So if we can move to smaller, equally effective models maybe we can afford to have them run continuously in inference mode.
One of the limiters to LLMs now appears to be data. Once you’ve Hoovered up the Internet, what other data is there that you can train on? So it might be that even if we added 10X more parameters the LLMs might not be much ‘smarter’ until we figure out how to train them more. One answer is happening now - hundredsmof millions of people are interacting with these things now, and all those interactions are data that will be used to train later versions.
That certainly wasn’t my intent, and I’m not sure what gave you that impression. Even the bit you quote clearly talks about how it is possible that such effects are relevant to consciousness, which is all the point I was making. That line of argument came as a response to @Dr.Strangelove’s claim:
I was merely pointing out that this isn’t as clear cut as it’s made out to be, because of the possible relevance of noncomputational dynamics to consciousness.
In the same way, I pointed out to @wolfpup that there are boundaries to emergence that we can know beforehand if the uncomputable has any relevance:
This is just going against the idea that basically whatever could just emerge from piling up more complexity.
So no, I’m certainly not intending for this to be a knockdown argument—on the contrary, I was merely arguing against premature certainties.
On the other hand, the proof provided in the other thread absolutely is a knockdown argument, that’s the nature of proof after all.
Fair enough. I guess I was getting a lot more certainty from your posts than was really there.
I can completely agree that there may be things about the way the brain works that preclude an AI from becoming ‘conscious’ the way we are, but at this point we don’t really have much evidence of that. But sure, it’s possible. For one thing, the brain does what it does on a few hundred calories of energy per day, while ChatGPT needed $30 million in CPU time for training, and costs about 3 to 9 cents every time you ask it a question because it’s so CPU intensive. Clearly the human brain does things differently. It’s also highly structured, and we don’t understand all of the structures and exactly what they do. And like all large neural nets, has stuff going on in it we don’t understand.
But so far, we keep getting surprised by these Large Language Models. They are exhibiting features that many cognitive scientists thought were either impossible for machines, or wouldn’t be achieved for decades.
I believe it was one of your cites in which someone in the field said that we were 50 years awasy from an AI that was good enough that we could even talk about it maybe passing the Turing test. And he said that in 2019. That statement was already invalid, and he didn’t know it. The transformer architecture was developed two years before.
A lot of the philosophy of mind that was developed in the pre-LLM era is going to have to be reconsidered. I would be wary of any sweeping statement made by a philosopher about what artificial intelligence can or can’t do that was written before say, 2021.
Except for questions of timelines, I don’t see how any of the arguments are challenged by the capabilities of LLMs. Can you give an example of what you have in mind?
Joking aside, Artificial Cunning is pretty much the default expectation for AGI, unless someone can solve the alignment problem (so far everyone who thinks they can solve it, isn’t coming up with anything that will actually work)
It seems completely unsolvable, honestly. In fact, humans becoming intelligent and creating AI that destroys us all seems like an alignment problem between the goals our genes had for us and our actual performance
Interestingly, I commented pretty much that under one of Robert Miles’ videos and it turns out he thinks it might be solvable, even though he says a lot about all the ways people think they have solved it, and actually haven’t. Or at least he hasn’t given it up as a lost cause.
Saw that a few hours ago in my feed. Here’s the conclusion for those who don’t want to click and watch:
A real problem with trying to make AI safe, especially once the systems start to get really powerful, is there are economic incentives to be fast, and people get stuck in this mentality, of like, “we have to be first.” Right? “We have to release our thing before our competitors release their thing.” And so, as a result, they neglect safety, they neglect alignment because alignment is fiddly, it’s tricky. It takes some time, and some money, and some engineering. And you can get this horrible situation where everyone would like to slow down, everyone would like to be more careful, but they feel like they can’t because they think that if they slow down, then their competitors won’t slow down, and so there’s kind of a race to the bottom when it comes to safety work. This is really, really concerning to me because if that pattern continues, things look really, really bad for us. If you develop AGI in this way, then there’s no hope of a good outcome ultimately because whoever gets there first is going to be whoever was being the most reckless. I want to be slightly more hopeful: humanity needs to step up its game a bit; we need to establish norms that are better than this because we can’t do it this way.
Because everyone has an incentive to be as fast as possible, all of the models will be limited by the hardware, which is not infinitely fast. Imagine instead that the companies working on this tried to be “safe” by only releasing very limited models. Then, a truly bad actor could gain an advantage by training/running their model on more powerful hardware. Or, a model that “escaped” would find itself with a great deal of room to grow immediately.
Since everyone is growing as rapidly as possible, we at least get a chance to address issues progressively instead of as a large step-function. We’re learning now the limits and best practices around alignment issues in a real-world way instead of via abstract thought experiment.
The first AGI will undoubtedly still be pretty dumb, and won’t have the potential for the unbounded exponential growth that people worry about (Skynet, etc.), because it will already be at the limits of its hardware. There will be some time before better hardware enables actual superintelligence.
I’m not going to claim that is a bulletproof counterargument, but I think it’s at least as convincing as the original claim.
While true, there aren’t orders of magnitude of performance left on the current hardware. There might be 10x or even 100x left in efficiency per transistor, but GPUs and dedicated tensor processors aren’t that configurable.
yeah, but I’m thinking more in terms of algorithms, and upthread there’s been talk of how sometimes smaller models can be trained to achieve the same results as bigger ones. What happens when an AGI is let to experiment on smaller models. Maybe it will try it on its own within its own neural net, and realize “Hey, I can chop off X, Y, and Z out of my current self and then run training steps sooo much faster.”
OTOH, I hold in my hand a tablet with computing power that might once have cost $30 million, weighed many tons, and occupied the entire floor of a building. I’m old enough to remember when the arrival of a new computer on campus necessitated the installation of a new power substation. The current physical nature of ChatGPT’s computing requirements does not by itself tell us that the brain necessarily does things differently. That said, it’s pretty clear that it does. Nevertheless, there’s no intrinsic reason that everything the brain does could not be emulated computationally, though probably in very different ways.
Apparently they have not rolled this out to all users yet, and not even all those of us paying for ChatGPT Plus. I tried asking it to do a search, it just responded with its usual “I am just a language model” boilerplate.
