Morality of Simcity type games

[msmith537]

Ok, now I'm a big fan of "god" games like Simcity and Civilization as well as "real-time strategy" games like Age of Empires, Command & Conquer and the like. I also realize the AI, while sufficient to simulate commuters or clashing armies, really isn't very sentient by any stretch of the imagination.

So how about in 50 years or so when we can create games/simulators with some rudimentary form of AI? The Sims (to borrow the Maxis term) experience some level of emotion make the gaming experience more realistic. They fear explosions and gunfire. They get pissed off sitting in traffic. They seek out things they enjoy. They experience indication of damage [ARNOLD T-2 VOICE]Vhat you hu-mans vood cahl paihn[/ARNOLD T-2 VOICE]

Is there a point where it would be "cruel" to set armies of self- aware virtual intelligences against each other for our amusement? Would the Command & Conquer 2500 experience be more or less entertaining if PTSD was a factor or if wounded virtual soldiers started crying for their momas while being ground under a N.O.D. tank?

Basically, if we create an AI that is indistinguishible from a human intelligence (or even a relatively smart animal intelligence) is there a point where it is no longer appropriate to use it as a soldier, acid mine worker or for any other purpose that we would consider cruel for a living intelligence?

(outcome of thread will not effect the fate of the residents of Simmsmithtown)

[Revtim]

This debate reminds me of the big flaw in Shelly's Frankenstein, as pointed out by Stephen King in his book Danse Macabre. The doctor makes a mate for the creature, but once he imagines the two mating and starting a race of creatures, he is horrified and destroys the female.

King points out that it would have been child's play for the doctor to simply to make the mate sterile; indeed, it would have been difficult work to make her fertile.

My point (I have one, believe it or not) is that once computer science is sophisticated enough to make computer characters that border on the conscious, it would likely be easy enough to make it so that they suffer no more than an actor suffers filming a death scene.

Of course, if they become conscious before we realize it, then that might be real bad news for a lot of Half Life 9 opponents....

[owlofcreamcheese]

well most people would still only play games for an hour or two at a time, so it wouldn't work to raise children up to warriors (unless game time was too fast to be a playable game). so it would require reuseing people, and so death would not be death as we know it. if a sim respawns a minute later its not the same as shooting a person with a shotgun... its more like a game of paintball.

[jayjay]

I know when I first started playing a game called Creatures, I ran across a few people online who seemed a little whacko about this already.

In the game, the way you train your creatures is by either kissing or spanking them. These people actually considered excessive spanking to be abuse, as in child abuse. Or at least their comments on message boards and game sites appeared to be sincere on this matter. I just kind of picked my jaw up off the floor and went on my merry way...

[DocCathode]

The great writer Stanislaw Lem did a story on this very topic.

A brilliant inventor meets an exiled tryrant. The inventor feels sorry for the tyrant, but realizes that the man belongs in exile. So he makes him a mechanical microkingdom filled with microsubjects. The tyrant is happy and the inventor goes home. He tells his best friend and fellow inventor about his creation. The friend is horrified and explains that with his genius and zeal for perfection, he has created not simulated people, but the real thing. The micro subjects have emotions, feel pain, truly live and can truly die. The inventor realizes that even though the microkingdom fits in a briefcase, all this is true.

Note, that the above description does not spoil the story anymore than revealing that Rosebud is a sled spoils Citizen Kane. The Cyberiad is filled with plenty of other fables on politics, technology and reality. I strongly recommend it.

[Master Wang-Ka]

It's already happened.

In more complex sim type games -- "Dungeon Keeper II," for example -- each and every individual creature in the game is represented by an autonomous little program and an autonomous little three-dee figurine, right? "Agents," in computer parlance.

We will, for purposes of this example, take the common Goblin, from DK2.

The Goblin will appear through your Portal as soon as you have any kind of support facilities -- Lairs and Hatcheries, for shelter and food. He will basically eat, sleep, and wander around doing nothing. Every so often, on Payday, he will wander into the treasure rooms and help himself to his pay.

Once you have installed a Training Room, this activates another thingy in his subroutine. He will go and Train there, beating up the little practice dummies.

If an Enemy (say, a Dwarf) wanders in, he will attack it. He will also attack enemy Keepers' creatures, but not yours.

You don't have to tell him to do any of these things. He's an autonomous little unit. You can use the Floating Hand to "slap" him, if you don't like what he's doing, or you can pick him up and put him where you want him (into battle, or the Combat Pit, or you can save him from a fight that's going wrong). Furthermore, if he's outnumbered, or has had the crap beat out of him, he'll retreat -- with or without your orders. He doesn't WANT to get killed!

Now, we have established that the Goblin is an autonomous unit, with needs and desires. We have also established that he has self-preservation instincts.

How much more complex does he have to get before it is morally wrong for me to slap him silly, toss him into the Combat Pit, or fling him into a melee with twenty angry Elves?

[foolsguinea]

I suppose you're against cockfighting, too. ::d&r::

[Kinthalis]

How could it be moreally wrong to slap him aorund, or smash him to bits Wang Ka?

Atleast at the current level of technology the little gobbo is nothing more than an encapsulated algorythm, nothing much more complex or alive than the iterating function used on this forum board to collect posts from a database and send them to our browsers.

[JasonFin]

Assuming we all agree that suffering is bad, the fundamental mystery you are touching on is the difference between simulated suffering and real suffering.

As simulations grow more and more accurate, this could become a very important question. Sometimes I worry that there isn't any important difference; that the suffering associated to an experience exists entirely in the eye of the beholder. We humans are simply programed to react to the suffering of other humans. When we see other beings as human, we will object to actions which appear to make them suffer. Some people have already crossed this threshold with current technology, as jayjay's Creatures example indicates.

[DocCathode]

Originally posted by JasonFin
Assuming we all agree that suffering is bad, the fundamental mystery you are touching on is the difference between simulated suffering and real suffering.


I really wish I had a copy of Cyberiad to quote from.

I agree with Kinthalis regarding Dungeon Keeper. The goblin is just a bunch of subroutines. If/then statements in the program react to variables. The goblin runs when the number of enemies is too great. It feels no fear.

[WaryEri]

I suppose this means I should stop deleting the kitchen door when my Sims light the stove on fire.....

[DocCathode]

Found it! (http://patricks.no-ip.com:8080/The%20Minds%20I%2018%20Trurls%20Kingom.htm)

"Trurl! Our perfection is our curse, for it draws down upon our every endeavor no end of unforeseeable consequences!" Klapaucius said in a stentorian voice. "If an imperfect imitator, wishing to inflict pain, were to build himself a crude idol of wood or wax, and further give it some makeshift semblance of a sentient being, his torture of the thing would be a paltry mockery indeed! But consider a succession of improvements on this practice! Consider the next sculptor, who builds a doll with a recording in its belly, that it may groan beneath his blows; consider a doll which, when beaten, begs for mercy, no longer a crude idol, but a homeostat; consider a doll that sheds tears, a doll that bleeds, a doll that fears death, though it also longs for the peace that only death can bring! Don't you see, when the imitator is perfect, so must be the imitation, and the semblance becomes the truth, the pretense a reality!

I agree with Lem that at some threshhold the simulation becomes a reality. We're nowhere near that point.

[swami]

Some people will not reconize the sufferings of animals (or at least a few animals), do you think this AI suffering will appeal to them?

[hansel]

Originally posted by jayjay
I know when I first started playing a game called Creatures, I ran across a few people online who seemed a little whacko about this already.I remember, a few years ago, a story on a guy who created "tortured norns": He would raise the creatures in an extremely abusive environment, tickling them into eating the poison plant, spanking them whenever they tried to play with the ball or eat something (but now and then tickling them towards real food). The norns were absolutely pathological--moving towards food, then away in fear, then closer as their hunger reached starvation levels, then away. They were the very picture of a hideously abused child.

This guy generated a level of hatred from the Creatures community that was unbelievable. They petitioned the company to de-licence his copy; they kicked him out of Creatures webrings; they downloaded his tortured norns and trained them back to health; they wrote long, angry screeds about his inhumanity. For his part, he said he was just interested about the parameters of the game, and only really got into it when he saw how much he was freaking out other people.

[Ramanujan]

i think the assumption that the OP makes is that intelligence defines what it is to be human and gives value to human existence. i don't necessarily think this is the case.

for one, we have no idea if plants might be super-intelligent but lack the ability to act on it, and do not show any suffering they might feel. vegetarians often prefer plants to meat because lettuce doesn't scream when you cut it up, but cows do.

the philosophical basis for the Turing Test is this: we observe consciousness in ourselves, and we exhibit a form of behavior that we see other people exhibiting, so we attribute consciousness to them, too. if there was a robot that gave out human-like responses, but looked like a big metal box, why would someone feel the same empathy for that as they would a flesh-and-blood human? it is not human, it does not act like a human, it does not look like a human. the empathy we feel for other humans stems from their similarity in many aspects to ourselves, and i don't think intelligence alone is enough to invoke empathy.

[Achernar]

Originally posted by Ramanujan
i think the assumption that the OP makes is that intelligence defines what it is to be human and gives value to human existence. i don't necessarily think this is the case.

for one, we have no idea if plants might be super-intelligent but lack the ability to act on it, and do not show any suffering they might feel. vegetarians often prefer plants to meat because lettuce doesn't scream when you cut it up, but cows do.No, I disagree. Self-aware intelligence is what defines being "human", by which I mean, "worthy of human rights". Your example with plants does not contradict this. The reason we eat plants and not dolphins is that we actually have good evidence and reasoning to believe that the former are not intelligent and the latter sort of are.

The continuum is actually really easy to see. The more intelligent something apparently is, the harder it is for us to kill it for trivial reasons. Plants, bugs, fish, mice, cats, gorillas, people. And if we found evidence that, say, mice are self-aware, this hierarchy would be rearranged in a heartbeat.

[Ramanujan]

Originally posted by Achernar
The continuum is actually really easy to see. The more intelligent something apparently is, the harder it is for us to kill it for trivial reasons. Plants, bugs, fish, mice, cats, gorillas, people. And if we found evidence that, say, mice are self-aware, this hierarchy would be rearranged in a heartbeat.i think your examples show that intelligence isn't the only thing being considered, but the ability to provide evidence for suffering. i don't claim that intelligence isn't a valued human property. my point is that human-like intelligence might not place a machine on the same level as a human, if the machine does not compare to humans in other ways. if humans could be turned on and off and repaired as easily as computers, it might not be considered so immoral to kill one. if machines could convincingly show that they suffered under certain conditions, those things might be more likely to be considered immoral.

[jayjay]

Originally posted by hansel
This guy generated a level of hatred from the Creatures community that was unbelievable. They petitioned the company to de-licence his copy; they kicked him out of Creatures webrings; they downloaded his tortured norns and trained them back to health; they wrote long, angry screeds about his inhumanity. For his part, he said he was just interested about the parameters of the game, and only really got into it when he saw how much he was freaking out other people.

While I think it's kind of twisted to do what this guy did, I just have immense difficulty suspending my disbelief enough to see the norns (or the Sims, for that matter) as anything other than a crafted representation of the programmed algorithms. Maybe that's why I never did get into the direct-command types of simulation games like Creatures and the Sims. I can only enjoy telling a computer program to eat or pee so often before it gets boring for me.

On the other hand, I get into the "governor" type games like Impressions City-builders much more. It's easier for me to get into a city-level simulation than to get into an individual-level simulation.

[Reply]

(Sorry for the long post... mostly just a bunch of gibberish so feel free skip to the next post if you want).

I think it would depend on the makeup of said artificial intelligence. IMHO, some questions to ask would be "Does the artificial intelligence in game XXX actually possess sentience? Does it experience emotion, like us?"

I disagree with what JasonFin said... I think that the actual presence of suffering is more important than a mere perception of it. If the "victim" does not actually suffer, then how can the act be cruel? True, we might perceive it as such, but that's just our perception... it doesn't mean the target of that abuse is actually experiencing any pain or suffering.

It simply isn't real, and I think that's important. Is it cruel when actors get murdered on screen? Or a character in a book gets tortured? It might look cruel, yes, but it's only a portrayal and not the real thing.

Computer games are the same way. Like other people have said, the AI in today's games aren't sentient yet and are just a set of predetermined responses to events. They experience no feelings and are incapable of suffering. We know this because we coded every line of them.

But of course, should AI ever advance to a level where it does genuinely experience sentience and emotion, instead of merely giving off the apperance of it as they do today, I think it would be cruel to mistreat them... simply because by then, they would be all too human.

When does it become morally wrong? When they can actually suffer.

But until/unless that day comes, I think it's all right to bash 'em all you want.

But what I want to know is... how will we be able to distinguish what actually feels emotions and what doesn't? What if we ourselves are nothing more than machines, programmed to go through a set of physical and mental reactions in response to stimuli? I don't know how our emotions really function, but what if they are just the biological equivalent to lines of code? "If I see big, bad object, then: send "afraid" signal to processor, which in turn makes heartrate faster, makes senses more alert, gets andrenaline pumpin', and then makes legs run the heck outta there." What does it mean to be self-aware? How can we test for it?

Apparent intelligence is not enough. What if a person was completely paralyzed? He wouldn't seem very smart, or even very alive, and would not be able to respond to much of anything. But is he self-aware? I would say yes.

In this case, things like brain scans might reveal thought activity, but what about forms of life that function completely differently? Who are we to say that thought and conciousness can only occur in the brain-type objects we are familiar with?

[msmith537]

Originally posted by Ramanujan
the philosophical basis for the Turing Test is this: we observe consciousness in ourselves, and we exhibit a form of behavior that we see other people exhibiting, so we attribute consciousness to them, too. if there was a robot that gave out human-like responses, but looked like a big metal box, why would someone feel the same empathy for that as they would a flesh-and-blood human? it is not human, it does not act like a human, it does not look like a human. the empathy we feel for other humans stems from their similarity in many aspects to ourselves, and i don't think intelligence alone is enough to invoke empathy. [/B]

But we can also feel empathy for non-human creatures. We feel little sympathy for ants because intellectually, they are about as complex as a "Sim". But we would not tolerate a person pitting an army of dogs and cats or other small animals against each other for amusement.

I have to agree with jayjay. The "Creature" was simply responding to a pre-determined set of algorithms that told it "IF ABUSE THEN LevelOfAgitation ++1". That's basically what any other videogame simulation would do.

If the opposing armies all of a sudden refuse to obey your command and sue for their own peace independent of your interaction, then I think it might be time to be concerned. ("um..yeah I want to return my copy of Total Anihilation 3000. They keep reaching some kind of armistice and refuse to fight!")

[shijinn]

[i]Originally posted by hansel
... they downloaded his tortured norns and trained them back to health ... you are kidding right?

[jayjay]

Originally posted by shijinn
you are kidding right?

Sadly, he's not. Creatures fandom is unexaggeratedly stuffed with people who can't tell the difference between a program sprite and a real animal...

[robertliguori]

Sadly, he's not. Creatures fandom is unexaggeratedly stuffed with people who can't tell the difference between a program sprite and a real animal...

I'll bite. WRT suffering, is a program sprite different than a real animal? Are scripted responses less valid expressions of suffering that nervous ones?

[Trinopus]

I'd be mopre worried about what it says about the person, rather than the sim.

For instance, what about someone who abuses stuffed animals? Ther is no question of a teddy bear feeling pain...but the sort of person who would knife one in a vicious manner is exhibiting signs of mental distress.

If the person in question subjectively identifies with the subject, to the degree of reacting to it as if it were independent and conscious, and if the person abuses that subject, then he is behaving in an immoral fashion.

Um... I need to add that some minor wickedness is "okay" in our culture... We can cheer for the defensive linebacker who puts the hurt on the other side's quarterback without descending into psychopathy... But if, say, one played a variant of a first-person football sim, and that was *all* one did -- qb sack after qb sack, specializing in the nastiest possible hits -- we'd have to look at that player with moral suspicion.

(Second the praise for Lem: "Talex of Pirx the Pilot" was full of scary exploration of the nature of consciousness.)

Trinopus

[ed]

I'll bite. WRT suffering, is a program sprite different than a real animal? Are scripted responses less valid expressions of suffering that nervous ones?

Consider the following script:

for(;;){
print "Ow, you're hurting me!\n";
sleep 5;
}

Is this program in pain? If not, what is the difference between this "scripted response" and the one you're hypothesizing about, other than (perhaps) more conditional statements?

If you DO believe this program is in pain, I'd like to understand why you think so.

[daldai]

I remember, a few years ago, a story on a guy who created "tortured norns":

This is good science actually, what this guy has done is test the roots of empathy, and whether true altruism really exists. These people who were angry with him were obviously pissed off at him because of how he made them feel, not because he caused any actual harm to any actual thing. Nice work fella.

[Captain Amazing]

Originally posted by DocCathode

I agree with Kinthalis regarding Dungeon Keeper. The goblin is just a bunch of subroutines. If/then statements in the program react to variables. The goblin runs when the number of enemies is too great. It feels no fear.

Well, but at its base, we're just a bunch of "subroutines" to, or at least the organic version. Certain nerves fire, and we call that "pain". Certain chemicals, adrenalin and such get produced in response to a stimulus, and we call that "fear". Why are our "subroutines" real feelings, and the goblin's subroutines not?

Not that I agree with the "Creatures" people, of course...slap around your goblin all you want...I'm not going to get outraged. (Just don't come crying to me when you go blind)

[Rhum Runner]

Originally posted by robertliguori
I'll bite. WRT suffering, is a program sprite different than a real animal? Are scripted responses less valid expressions of suffering that nervous ones?

Yes, and yes.

[DocCathode]

The goblin's coding tells it what to do not what to feel.

The goblin runs away and looks afraid. It does not get scared, and flee in terror.

Importantly, IMO anyway, the goblin has no spontaneous emotional response.

He doesn't flee in terror when the odds are in his favor.

He doesn't suffer post traumatic stress disorder after battles.

There are no goblin conscientous objectors.

[robertliguori]

The goblin's coding tells it what to do not what to feel.

The goblin runs away and looks afraid. It does not get scared, and flee in terror.

Importantly, IMO anyway, the goblin has no spontaneous emotional response.

He doesn't flee in terror when the odds are in his favor.

He doesn't suffer post traumatic stress disorder after battles.

There are no goblin conscientous objectors.

Doc, to a certain extent, all those things apply to the creatures referenced above. Does it make a difference?

[Master Wang-Ka]

...and there's my question.

First of all, I am not interested in the motives of the guy torturing his Norns, nor am I interested in the sapience of the creatures, although that may become a later consideration.

The question here is this: is it wrong to torture and abuse simulated lifeforms or not?

We have laws against cruelty to animals. This means that we, as a society, feel that it is wrong to inflict needless suffering. If one MUST have a hamburger, one should raise the cow in a humane fashion, kill it as fast and painlessly as possible, and make use of it in a nonwasteful manner.

Bear-baiting, on the other hand, is a terrible, disgraceful thing, under this particular moral code, and should not be countenanced by civilized people -- since it basically involves torturing helpless animals to death for fun.

So, from here, we go into whether or not it's okay to torture SIMULATED animals to death for fun.

On one level, this is easy enough. Remember Teddy Ruxpin, the talking teddy bear? Simply remove his story cassette, and insert a different cassette, one which has a person screaming in agony. Activate Teddy Ruxpin, and suspend him over a slow fire. There we go, torturing simulated animals to death for fun.

Is this wrong? No. Teddy has less awareness than a flatworm. Teddy is a stuffed animal with a tape recorder stuck up his ass. There is no moral quandary here.

On the far end of the spectrum, we have sapient artificial beings, like, say, Lt. Commander Data from Star Trek, or Andrew Martin, the Bicentennial Man (from the short story and film of that name). Data and Martin were both artificial beings, but capable of feeling and desiring and even understanding and articulating.

At one point in the book, a gang of street kids intercepts Martin on his way to the library, and orders him to disassemble himself. Martin is trapped by the Laws Of Robotics; he MUST obey a direct order from a human, except where such order would involve HARMING a human... and he must take himself apart, despite the fact that he has self-preservation programming... and is sapient enough to realize that he has been ordered to kill himself, a situation very much against his will!

This ultimately leads to a court decision: "It is immoral to deny freedom to anything capable of understanding the concept... and desiring it for itself."

Well, this would seem to imply that I am a bad person for keeping my idiot cat in the house so she won't run out and fling herself in front of a car while trying to get it to pet her, but this would seem to also be applicable to Norns... or DK2 Goblins.

It is true that Norns and Goblins are nothing but pixels and subroutines, sure.

But I, Wang-Ka, am nothing more than a heap of meat with a variety of subroutines geared to ensure that I will live a while and make more little Wang-Kas, yes?

Subroutines like "fear," "pain," "pleasure," "joy," "horniness," and so on, motivate me and control my reactions. The fact that I am sapient does not alter these subroutines or their effects on my behavior; it simply alters how I rationalize them, and to some extent, how I act on them. The fact remains that if you offer me gold, food, or a big wad of money, I will motivate towards you. If you pull a .45 or sic a gang of Elves or Grendels on me, I will react yet another way.

SO: we have established that Norns and Goblins and Sims are not sapient... but I have seen them act autonomously to meet their own needs and desires. They are, perhaps, less sapient than a cockroach, but they have needs and desires and motivations. They like some stuff, and don't like other stuff, and their behavior reflects that.

So... um... is it wrong to step on cockroaches? Is it wrong to torture one to death for fun? And is a Sim or Norn more or less than a cockroach?

[Ramanujan]

Originally posted by DocCathode
The goblin's coding tells it what to do not what to feel.perhaps your point would be made more clear if you could explain the difference between what someone feels and how he behaves.

why is the conclusion that there is actual pain when a person says "ouch" but not when a computer says "ouch"? is it because we don't know what causes the person to say "ouch"?

if one takes the position that a pin prick can cause a person to say "ouch", and that that person feels pain, perhaps the holder of that position could explain why a machine that says "ouch" when it is pricked with a pin does not feel pain.

[eburacum45]

How cruel, then , to create creatures that can feel pain, and love, and hate, then demand that they love us.
And punish them if they don't.

If we can't do it now, then one day it will be possible.

Secondary creation; dontcha just love it.
__________________
SF worldbuilding at
http://www.orionsarm.com/main.html

[Vriggs]

The program is a simply a facsimile of a living creature.

Will we eventually be able to create an android that is our perfect mate and we can fall in love with (i.e. program her to be exactly what we want - intelligent, love sports, slutty in bed etc..) and look exactly how we want it to look? It becomes scary if anything and I want no part of it... not to marry at least .. date maybe.. hrmm

Anyways to the OP, I think the only way it is detrimental to us as humans is in how real we perceive these creatures. To be cruel to them without the absolute clarity in your mind that these are not real must have some detrimental effect on your humanity.

For those of you who picked up on it, it's the same opinion Kant has human treatment on animals... I don't know if growing up in the society I have I can agree with him re: animals but certainly re: sim games. Am I conceeding cultural relativism? Yes.

[owlofcreamcheese]

I'm takeing a class in AI this semester. take a class on AI and see its all A* searches and hillclimbing algorithms and monkeys climbing ladders and you will see this is not a topic that really will be relivent for a long time.

beyond that there is no reason to belive that an artifical world would be 1 to 1 translation of our world.

say you wanted to make a quake 22 player that had human quality AI. to make the AI the embodied in the enemy would produce an unfun game. it would fear pain and run and hide, it would not fight in a way thats amuseing, it would not make crazy leap 50 feet in the air attempting to shoot you and land on the tower that has the BFG. it would be much better to make the AI another player... a controler of a game charactor... not a creature embodyed in the game.

nearly any game would work better with this abstraction. the actions a sentient creature embodied in the game would not be fun to play for 30 minutes or an hour... they would have all sorts of daily life stuff to attend to... or would run away too much or whatever... haveing them as an abstraction of a player or a controler would produce a far better game in almost all situations.

[Ale]

Subroutines like "fear," "pain," "pleasure," "joy," "horniness," and so on, motivate me and control my reactions. The fact that I am sapient does not alter these subroutines or their effects on my behavior

You´ve forgotten that thing about will, we don´t just react to stimuli in a IF xxxx THEN xxxx fashion, that´s part of beign human and I suppose of beign sentient; a really sentient goblin may not run away in front of an overwhelming enemy, perhaps he would charge if a sense of duty strikes he. What I´m trying to say is that IMO one of our most important characteristis is our caotic response to stimuli.
Then again a programmer could add that behaviour to a virtual creature... :confused:

[Lamia]

Originally posted by jayjay
Sadly, he's not. Creatures fandom is unexaggeratedly stuffed with people who can't tell the difference between a program sprite and a real animal...

Okay, I'll bite. I never played Creatures myself (I was a Civ II woman in those days), but my baby sister was a big fan. She found the "tortured Norns" site and downloaded some to nurse back to health and happiness. Why? Mostly because she wanted to see if she was good enough at the game to be able to do it. She had already raised plenty of well-behaved Norns herself, and I guess she wanted a challenge. I don't see why that's any weirder than wanting to see if you can cause the game characters to manifest deviant behavior in the first place.

[Tusculan]

"But I just wanted to turn off Skynet".
"Suspect confessed intent to murder."

Seriously, though, if we are discussing the morality with Norns and Sims, shouldn't we extend (logically speaking) the same reasoning to less cuddly types? I don't hear people complaining about ending Skynet, turning off HAL (who did seem very human). Nor is there much complaint for the sake of the game characters about shooters like Doom, Quake, Half-Life etc. The designers actually talk about the 'pain' or ' hurt' these creatures are able to withstand.

I do not find it problematic per se that the game character hurts. What I do find to be troublesome is the attitude of the player: if you take pleasure in suffering of others (which seemed to be the case with the Norns), you would have a serious moral shortcoming. Notice that in most shooters the set-up is one of self-defence. Í do remember a game (Rise of the Triad?) that had the 'enemy characters' actually beg for their life when you tried to finish them off. That made me dislike the game: I found it repulsive having to shoot a character in such a situation.

[shijinn]

thanks Lamia, that makes sense. the context given by hansel imply doing it out of spite. Originally posted by Wang-Ka
... Subroutines like "fear," "pain," "pleasure," "joy," "horniness," and so on, motivate me and control my reactions. ... Originally posted by robertliguori
I'll bite. WRT suffering, is a program sprite different than a real animal? Are scripted responses less valid expressions of suffering that nervous ones? unless we are able to program the software equivalent* of pain and suffering (as opposed to behaving as such) then the spotlight will not be on Teddy Ruxpin, but on the said torturer.

* example data from star trek 'first contact' - you can mutilate him all you want, and he might react in pain if he want, however it is not suffering unless he turn on the pain receptors.

[Badtz Maru]

Originally posted by owlofcreamcheese
I'm takeing a class in AI this semester. take a class on AI and see its all A* searches and hillclimbing algorithms and monkeys climbing ladders and you will see this is not a topic that really will be relivent for a long time.

beyond that there is no reason to belive that an artifical world would be 1 to 1 translation of our world.

say you wanted to make a quake 22 player that had human quality AI. to make the AI the embodied in the enemy would produce an unfun game. it would fear pain and run and hide, it would not fight in a way thats amuseing, it would not make crazy leap 50 feet in the air attempting to shoot you and land on the tower that has the BFG. it would be much better to make the AI another player... a controler of a game charactor... not a creature embodyed in the game.

nearly any game would work better with this abstraction. the actions a sentient creature embodied in the game would not be fun to play for 30 minutes or an hour... they would have all sorts of daily life stuff to attend to... or would run away too much or whatever... haveing them as an abstraction of a player or a controler would produce a far better game in almost all situations.

I can see the problems a real thinking and feeling AI would present to the fun of FPS games, but there are plenty of other types of games where having self-aware beings that thought they were really alive would be a great enhancement - like The Sims, or action games with lots of civilian bystanders.

[Master Wang-Ka]

How cruel, then , to create creatures that can feel pain, and love, and hate, then demand that they love us. And punish them if they don't.

Hm. Makes me wanna go back and reread the Old Testament, don'cha think?

And while it's true that a Goblin is simply a buncha "If situation X, then reaction Y," subroutines... how is a cockroach any different? I have yet to see a cockroach or earthworm which would fling itself on a bug bomb to save its comrades.

Seems to me that as far as programming and subroutines go, there IS no difference between a Goblin or Sim and a cockroach... which would seem to bring up a morality issue, if you feel that killing roaches is wrong.

For fun, at least.

[alterego]

Are these simulations moral agents?

If they are not moral agents then we could probably justify subjecting them to the same tortures that we subject animals to. Including suffering and poor living conditions, as well as death. In most western society it is not considered right to pit animals against eachother for human pleasure.

If they are moral agents then they deserve the rights of a moral agent.

[Alan Owes Bess]

I believe that in the final analysis, in order to judge the morality of a game, one should first take a serious and sober look at the underlying purposes and objectives of the game.

My first experience in this genre, or, for American readers, gener, was a simplified version of "Simcity", possibly even a forerunner, called "Simbrick".

In this game, the information a player could call up for guidance rovided a diagram giving the precise dimensions of the brick in both imperial and metric units.

The action part was the most exciting and depicted a brick falling on a large inoffensive ant from a great height, with predictable and bloody results.

The pointless killing of the ant troubled me greatly so I refused to play this version of the game after a few sessions at the expert level.

[Bryan Ekers]

Well, all of this is countered by the game Lemmings in which you desperately try to save the little buggers from themselves. But for your involvement, they would gladly march off cliffs and into meat grinders and whatnot.

Of course, after a while, increasing frustration with the game often leads to a feeling of "Let the little bastards die!"

[msmith537]

Originally posted by Vriggs
Will we eventually be able to create an android that is our perfect mate and we can fall in love with (i.e. program her to be exactly what we want - intelligent, love sports, slutty in bed etc..) and look exactly how we want it to look? It becomes scary if anything and I want no part of it... not to marry at least .. date maybe.. hrmm
[/B]

Hey I get enough of that lovie-dovie stuff from my girlfriend. I just want a hot chick naked chick to paint my house and serve drinks when I have guests over!:D



Alan Owes Bess - That game does sound pointless and cruel (unlike Age of Empires II where my valiant soldiers sacrifice for the greater glory of the msmith537 empire!)


It does seem strange that we take so much pleasure from seeing scenes of destruction and gore in movies and VGs that would horrify us in real life. People were not cheering the collapse of the World Trade Center (unless you were a guy with a ratty beard named 'Ahmed'). But the same structure being destroyed onscreen by Jerry Bruckheimer with a 1920 Style Death Ray is guaranteed to make the audience cheer.



"On the far end of the spectrum, we have sapient artificial beings, like, say, Lt. Commander Data from Star Trek, or Andrew Martin, the Bicentennial Man (from the short story and film of that name). Data and Martin were both artificial beings, but capable of feeling and desiring and even understanding and articulating."

I have to wonder though (and maybe this was actually covered in a ST episode), if push came to shove, would Data's "life" be as valuable as another crewmans (not counting "red shirts")

[Chicago Faucet]

Whenever my wife plays one of those simulation games, it always ends the same way. After the novelty wears off, she gets bored and starts devising creative ways to kill off her sims.

In one respect, this is simply human nature: growing jaded, and looking for a new source of entertainment.

In this sense, my wife is acting like an AI, controlling other AIs - as I see it.

[eburacum45]

Ah well, when simulations and virtuals achieve full self awareness, it will be necessary to afford them some sort of sentient rights;

simulations have the advantage that they can be backed up more easily than us fleshies, and so can be killed without too much inconvenience. As responsible virtual citizens (http://www.orionsarm.com/sophontology/virtuals.html) they will have to make sure their backups are up-to-date themselves.

[Bytegeist]

(1) Another computer game that provides hours of simulated death and carnage, in gory squishy detail, for hundreds of hapless simulacra under your irresponsible control, is Bungie's Myth series — which I highly recommend. (Myth I and II are long out of production, but you can find them cheap in many computer stores' discount bins.) However, as in many games, the combatants don't really express any suffering or aversion to death. They move and fight and charge into battle just the same as always right up to the moment when their health falls below zero, and they quickly expire.

I'm sure that's not an accident. As other posters pointed out, a game in which the characters did suffer and scream, or beg pathetically for their lives, would cast the human player as a psychopath. Most of us don't want to be made to feel that way, especially in a game we bought with our hard-earned cash and are using for recreation. And I'll bet most parents don't want to overhear such sounds coming from their teenaged boys' bedrooms, where these games are more commonly played. (At least I hope "most" people are like this. I don't have a supporting study to back it up I'm afraid.)

However, the characters in Sims or Myth are more like chess pieces with some special effects. I know I'm fond of Myth's impish grenade-tossing dwarves and their deranged cackling, but I also know that they're all identical, they have no history or future, and that the game will generate another batch of them whenever I restart the level.

(2) On the other hand, there's the game Rune, a first-person 3D action game I'm also fond of, but which includes a feature that made me cringe with remorse the first time I encountered it.

You are a Viking in a medieval world of magic and demigods and ghoulish creatures. In several of the levels, you'll find yourself running down castle corridors holding a flaming torch for a light source (and occasional weapon). The evil masters of the castle have hung some human prisoners upside down from the corridor ceiling. There's nothing you can do for them; you just have to run by and leave them hanging. They are part of the scenery.

This by itself isn't so bad. It adds to the game's creepy atmosphere. But if you bump up against a prisoner while holding your torch, no matter how briefly, you will set him on fire and he then howls in agony while aflame. Thankfully he doesn't convulse; I don't think I could take that. As it is, I always apologize before moving on. That's just the way Momma raised me.

(3) A good book that explores "artificial suffering" a bit is The Mind's I by Hofstadter and Dennet, which (wouldn't you know it) includes an excerpt from Lem's Cyberiad as a starting point for one chapter's discussion. The book is full of philosophy essays and short stories on artificial intelligence in general, how it might be built, how you'd recognize it if you found it, and the ethics of dealing with it. The book is a thought provoking read, even if it never answers anything very concretely. Consciousness is a pretty slippery topic after all.

[Vriggs]

Isn't this whole ethical dilemma played out on Star Trek with Data?

We are constantly left guessing as to what level of humanity he possesses and are constantly struggling with the question of whether he deserves the same rights as the rest of the crew. In fact I believe there was an episode directed at this very point when a scientist of some sort wanted to 'open him up' and investigate his android anatomy.

I think the concencus with Data is that it is wrong to harm him - but lurking beneath as to why - imho it's because we perceive him as human.. so to harm him we are in fact saying it is ok to harm another human.

Sorry if this is a hijack..

[Ramanujan]

Originally posted by eburacum45
simulations have the advantage that they can be backed up more easily than us fleshies, and so can be killed without too much inconvenience. As responsible virtual citizens (http://www.orionsarm.com/sophontology/virtuals.html) they will have to make sure their backups are up-to-date themselves.something i wonder when considering the nature of consciousness is, how is this different from human beings?

cloning is no longer a thing of the distant future, and if we came up with way to "save" ourselves, would that change our morality with regards to human life?

i have to believe it wouldn't. at least not significantly. the reason i feel this is that though for all appearances to the outside world, this "saved" copy would be me, i firmly believe that it would not be me, that this consciousness would end the moment anyone tried a "revert to saved". i feel like if i was threatened with death, knowing that there was a perfect replica of me that would have exactly the same thoughts and memories, minus the death experience, this would not make me any more comfortable with death. bearing that in mind, my human empathy makes me feel that most others would feel the same way and i would not wish them to go through the discomfort of death, even if i wouldn't feel the loss associated with it.

so i'm not sure having a backup copy of a simulated human would grant us any greater freedom with regards to the morality of its termination.

[shijinn]

Originally posted by Ramanujan
... so i'm not sure having a backup copy of a simulated human would grant us any greater freedom with regards to the morality of its termination. ... that is debatable. i assume your thinking will mean you will object to the use of teleporters (http://boards.straightdope.com/sdmb/showthread.php?s=&threadid=203463) as well. placed in that context, where using a teleporter would be no different than using a 'saved game'; there seem to be a divide between those who share your thinking and those who would sacrifice themselves for the sake of convenience.


can a cockroach feel pain?

[jayjay]

Ramujan, that's not the way clones work in the real world. If you were cloned, the clone would be your younger twin, not another you. There is no way to "transfer" your consciousness to the clone. And even if there were, the clone wouldn't be some kind of tabula rasa that you need only scribe your memories onto to activate. It would be its own person, growing up quite differently from the way you did (unless you have a Boys From Brazil program in place, anyway) and therefore turning out a different person in the end.

Only in bad science fiction are clones possible full-body replacement units.

[Ramanujan]

Originally posted by shijinn
that is debatable. i assume your thinking will mean you will object to the use of teleporters (http://boards.straightdope.com/sdmb/showthread.php?s=&threadid=203463) as well. placed in that context, where using a teleporter would be no different than using a 'saved game'; there seem to be a divide between those who share your thinking and those who would sacrifice themselves for the sake of convenience.


can a cockroach feel pain?
you bring up an interesting point about teleporters. i'm not sure if it works out the same though, since with teleporters, it is presumably the same molecules and such that make you up on either end, so it could still be said that the teleported version is still "you". perhaps.

my point is that we need a better understanding of what consciousness is before we can tackle these sorts of questions.

as something more to think about, suppose you were having your brain replaced by computer bits, piece by piece. suppose, to further complicate matters, that you were awake and aware the whole way through. one would expect the first neuron replaced by an implant wouldn't affect you terribly. but after the procedure was completed, there would be nothing left of your brain. it would be a different brain. the question, then, is at what point would you cease to be you?

[Ramanujan]

Originally posted by jayjay
Ramujan, that's not the way clones work in the real world. If you were cloned, the clone would be your younger twin, not another you. There is no way to "transfer" your consciousness to the clone. And even if there were, the clone wouldn't be some kind of tabula rasa that you need only scribe your memories onto to activate. It would be its own person, growing up quite differently from the way you did (unless you have a Boys From Brazil program in place, anyway) and therefore turning out a different person in the end.that would be why i spoke of "saving" rather than cloning, giving cloning as an example that this sort of idea might not be something we don't have to think about.

Only in bad science fiction are clones possible full-body replacement units.for the moment.

[eburacum45]

That's right- the process is-
upload your mind into electronic media- c
opy that upload as many times as you can afford-
earn your virtual keep by working as a bit part player in a shoot-em-up sim;

get killed whenever the sim scenario calls for a realistic death-
download your surviving self (or selves) into (a) realistic robot body(ies) whenever you have gathered enough virtual credits.

Such a body could be very realistic in the far future; but why limit yourself to a realistic body.

(oh, and this is probably a far too conservative view of the real future, which will be much stranger).
__________________
SF worldbuilding at
http://www.orionsarm.com/main.html

[owlofcreamcheese]

I still say its a silly way to do it to embody any AI in a game, nearly any game would be better with the AI opperateing at a control level. in that way its like playing against a person, who reacts useing the rules of the game, not the rules of the world of the game.

no game would be fun if you inflicted real pain on the charactor, not for some moral reason that thats ghastly... just in the fact that once you shoot the guy once hes not going to play right anymore if the other player is playing for their life they will play in a way that is not so much fun as it is trying to stay alive.

think of it this way, if you invented AI and wanted to make a chess game the worst way to do it would be to give the peices AI. the better thing to do would be to give the AI to a second player who could move the chess peices.

to embody the AI into a virtual charactor in the game world makes for a stupid game in almost any case, most games its not a problem of the enemys being not smart enough, quake could make enemys that could aim perfectly and calculateing the travel time of rockets and everything and beat nearly everyone every time. thats not fun though, what all the programming goes towards is makeing bots that play like a person would play. not a bot that is better at the game.