I had never heard of Cleverbot, so I went and Googled it. Here is my first converstation:
96867 people talking
Me: Hello, who are you?
Cleverbot: Hello, I’m your friend :).
Me: Do you like reading books, friend?
Cleverbot: Yes, especially books about the mind.
Me: Do you think I would like one of those books?
Cleverbot: Do you know anything about eating humans?
Reading that snippet, I’m not even sure if they’ve improved on ELIZA. You can’t hide a prescripted non-sequitur question just by adding emoticons.
On the other hand, I’m sure the code behind the responses is much more interesting than the results it produces.
Merged duplicate threads.
They’re fun. (genuine cleverbot response)
Indeed. I guess “33% of Humans Fail Intelligence Test” doesn’t have quite the same headline appeal.
Tell me more about what specifically.
Would you like to play a game?
Color me thoroughly unimpressed. The Turing Test is something I’ve been fascinated with for years; hell, Alan Turing is one of my idols, and this just looks all around like an attempt to fudge the numbers to hope to have something publishable. To my knowledge, the test as proposed by Turing was always vague and never included five minutes of discussion and 30% success. Also, why a 13-year-old boy who speaks English as a second language? Of course it’s easier to fool people with bad grammar when pretending not to speak the language natively and pretending to be someone fairly young. Hell, I bet I could get an even bett success raid by having a system pretending to be a 4-year-old girl from Uganda.
In my view, there is no “pass/fail” for the Turing test as proposed. Rather, we’d just state certain benchmarks, like after x minutes of conversation it passed as human y% of the time. I guess one could argue that it has meaningfully passed when it can have a sufficiently long conversation, say on the order of 30 minutes to an hour and convince a large majority that it’s human. Regardless, the whole point of the test is a philosophical argument that we judge intelligence based on how it compares with humans. At best, we can say here that this demonstrates a single experiment in which their system is seen as more likely to be a 13-year-old Ukrainian boy speaking English as a second language 33% of the time.
And, quite frankly, looking at the transcript, I really have no idea how anyone would think that’s from a human. Really, I’d be much more interested in not just which the judges think is more likely, but also what they weight their confidence at. Was the judge that thought it was human 70/30 or 55/45? I’d suspect close to the latter than the former.
Yeah, I think the Turing Test is more a thought experiment and springboard for discussion than an actual “test” that can be taken with pass/fail results. Almost certainly, “passing” a Turing Test doesn’t get one anything specific in terms of awards or certifications, except with respect to specific programming competitions that have more specific or restrictive rules for competition.
Right, the “point” of the Turing Test, is that if you can’t tell the difference between a conversation generated by a thinking human being and a computer program, what does it mean to deny that the computer program can “really” think?
It is intended to deny the distinction between “real thinking” and “fake thinking”. If you can fake thinking so well that you fool everyone into believing that you can really think, then it is perverse to call that fake thinking.
That said, a computer program that emulates a rude illiterate 13 year old boy who barely speaks english for five minutes isn’t much of an accomplishment. OK, so we expect a 13 year old boy to spout non sequiturs, not know anything about anything, not remember conversations from 2 minutes ago, and so on. You could do the same thing with Eliza. Some people are really boring, can’t type, can’t be bother to carry on a chat session. If I wanted to emulate my mother in law it would be, “Cn’t type. Keboard semz brooooken who do I crocct wors i have tyeped? Dang comupters! Talk to you next week kthxbye.”
I finally got a chance to try it. (Their website has been slammed.)
It fails pretty obviously and spectacularly with any sort of meta question – things like “What do you mean by that?” or “Why do you think so?” It’s basically Eliza with a bigger database and some misdirection tricks when it gets stuck.
I’ve coded up a chatbot that is indistinguishable from a real 6-month-old Indonesian girl. What do I win?
Selamat pagi, kawan!
You have a case of long-term memory loss, or are too young to have spent an evening keying in ELIZA.BAS, only to have your data storage system/$20 cassette recorder eat the file. Or tried to convert it to Sinclair BASIC, which didn’t have DATA statements.
An interview with Chris Hansen.
What? We were all quoting the WTF segues bots like Eliza and Cleverbot make.
What does that suggest to you?
Techdirt has a good article about this. “No, A ‘Supercomputer’ Did NOT Pass The Turing Test For The First Time And Everyone Should Know Better”
Among other things, they point out that this was organized by Kevin Warwick, who claims he’s a cyborg because he had a chip implanted in his arm.
I wrote a program that passed a similar Turing test years ago. I told the judges that the person they were communicating with was a senile old man with a head injury and motor skills too poor to operate a keyboard, and then spewed gibberish at them. Their guesses were about 50/50.
The public understanding of the Turing test is really dumb anyway. I say public understanding because I don’t know if it accurately reflected anything said by Turing, but the idea that using this sort of technique to identify artificial intelligence is silly.
We could very well create an artificial intelligence that’s unlike humans entirely, and yet has its own intelligence and personality. But it may not communicate in the same way that humans do, and therefore could easily be identified. That wouldn’t preclude it from being an artificial intelligence.
In the other direction, you can fool people with elaborate scripts that aren’t in any way even close to being an artificial intelligence.
You may as well just read the initial paper. It’s not a particularly difficult read.
Turing starts off by proposing a game called the “imitation game”. A game with a man, a woman, and an interrogator. The goal of the game is for the man and woman to fool the interrogator about their gender (done in an anonymized way such as by typewriter to control for things like tone of voice or handwriting).
He then goes on to more or less say that since humans are relatively good at playing the imitation game, a good test for machine intelligence may be that it is also about as good at some trivial variation of the imitation game as a normal human. By my memory, he never claims being able to pass this test is necessary, nor sufficient for true AI. Simply that it’s a valid metric of one component of intelligence.
Perhaps a better analysis of the original Turing Test would be that an intelligent computer should be, on average, roughly as good at pretending to be human as a human is at pretending to be an intelligent computer.
Except there are some things that an intelligent computer should be able to answer in milliseconds that would take a normal human being minutes or hours, like “What’s the cube root of 58493.9456?” Of course, if you give the human being a calculator app on the laptop they’re using to play the game, then they can punch in numbers and get back results just like a computer.
Of course if you expect the computer to parse a word problem, identify the values and the equation being asked for you’d see different results again. But many humans aren’t able to do this either. “A train leaves New York traveling 95 miles an hour…”