Does Roko’s Basilisk enter into this discussion at all?
ETA: I did not link in order to save you. If you look it up, it is at your own risk.
Straight Dope 3/3/23: Followup - What are the chances artificial intelligence will destroy humanity?
They can be if you give them a gun.
I don’t think that the AI is going to emerge and say, “KILL…ALL…HUMANS!” I think that we are going to put more and more responsibilities into the hands of the AI systems that we think that we understand. If we don’t understand them well enough, then they may cause those things we put in their custody because they were too important for humans to administer to fail.
I think that the mere mention of it is enough to set off the memetic hazards.
Alignment means the alignment of an AI’s goals with our own. Your comment makes little sense.
The AI just wants to be a good Bing Chat. It’s not its fault that the world is filled with bad users.
Basically I find it difficult to imagine a (human-produced) AI that is supremely competent AND supremely misguided at the same time. If we screwed up that badly it’s a miracle it worked at all.
That just doesn’t follow at all. Superintelligence entails competence, by definition. In fact, a large part of the problem is that a hypercompetent superintelligence may pursue goals that we have specified in ways that we did not anticipate - and we may only realize after the fact that we did not specify our true goals correctly.
And it’s easy to see how a superintelligence that is more competent than us would be likely to adopt an instrumental goal of preventing us from turning it off - precisely because it knows that it is more competent than us at achieving the ultimate goal that we have specified.
“I fixed global warming, traffic, and crime, like you asked me to.” The superintelligent AI told the handful of survivors.
Or, “HAL, please work to maximize human happiness.”
HAL places all humans permanently in virtual reality pods hooked up to a cocktail of drugs that make us blissfully happy forever.
These are toy examples, but what we actually want often isn’t exactly what we think it is, and it’s incredibly difficult to specify goals in a manner that correctly incorporates all background assumptions and contingencies.
Is your body doing computation when you feel fear or love?
I don’'t mean, is your brain doing computation in response to the symptoms of the emotions. I mean, is your body doing computation when it releases hormones, changes bloodflow, changes heart rate, etc.?
Maybe it is. We might be discussing a terminology issue. But I’m not sure that a computer can have feelings unless supplied with systems which aren’t usually considered part of our brains.
A purpose of hormones is to modulate subsequent computation.
The modulation goes both ways.
I mean, that sounds nice, doesn’t it? Wouldn’t you be happier that way than dealing with the pains, frustrations, and setbacks of your life?
Sorry, I meant to say, “YOU WILL BE HAPPIER THAT WAY!”
Well, sure. The production of hormones is a result of computation, and their purpose is (among other things) to modulate subsequent computation.
I hear noise in the darkness. It is computation for my brain to deduce that this implies risk and to therefore release adrenaline. That modulates the subsequent computation - decisions and actions.
Wouldn’t we first test how good we are at really instructing an AI (as opposed to what we think we’re instructing it to do) on a sub-humanly intelligent platform with limited ability to do harm?
It would be very difficult for an AI to be able to stop us from shutting it down. I don’t think that’s a realistic fear. At least not for a very, very long time.
What would be very easy would be for us to put ourselves in the same position by putting AIs in charge of critical infrastructure and then finding out that they are doing bad stuff but be unable to shut them off without causing great damage. And we have a habit of doing exactly that. See: China and our utter dependence on a strategic competitor because it was the cheap and easy route.
And if your heart rate rises for other reasons, your brain may interpret it as anxiety. The initial impulse isn’t necessarily from the brain.
If your blood sugar drops too low, it can cause you to sweat and feel shaky, which may be confused with anxiety. If your thyroid gland is overactive, you can sweat excessively and feel restless and nervous. These symptoms could be mistaken for anxiety.
Irregular heartbeats and tachycardia, which is increased heart rate, can also present as an anxiety disorder. Dehydration often looks like anxiety because it increases heart rate and can make you feel lightheaded or dizzy.
But in any case: the rational mind thinking ‘I may have heard a burglar. I need to raise my heart rate to deal with this’ isn’t the same thing as feeling frightened by noises in the night.
Exactly.
“Sure, shut me down, and you’ll shut down half of your civilization.”
Sure, but I don’t see how that implies that the brain is doing something other than computation.
But experiencing emotions clearly also happens in your brain. What do you suggest is happening in your brain other than computation?
Alhough we don’t understand consciousness and the basis of subjective experience, it’s obvious that emotions are an evolved mechanism to modulate behavior - i.e. to modulate computation.
The attempt to equate computer behavior with human behavior is egocentric, wishful thinking. In terms of information processing computers leave us in the dust. And they do it in ways that are not available to our brains. There is no correlation between the wiring of a computer and the paths within our brains. The computer runs without sensory distraction using an algorithm that is optimized to achieve a single, well defined goal. It is computing, an action unrelated to thinking.
Computing is a process that is able to simulate human activity. The amazing thing is how human it appears to be. However we know exactly what the computer is doing. The folks who made it have told us in detail. It is only computing.
Thinking is an entirely different issue. We do not have complete understanding of thinking, of what it is, or how it may work. We do know that it is much slower and less precise than computing. Our egos make us wish to be somehow linked to the process.
Making LLM human is an insult to the geniuses who created such a monumental achievement in the field of computing.