I’m having fun with an artificial intelligence toy that plays the old children’s game “20 Questions” (sometimes called “animal,vegetable, mineral”).
You probably know how the game works: You decide on a topic, for example: book, telephone or cat. Then the toy asks you 20 questions, you answer “yes” or “no” and then the toy miraculously tells you what topic you chose.
The toy has a series of maybe 100-200 questions , such as “do you hold it when you use it?”-- “can you put items inside it?”-- “is is colorful?”-- “Will it fit inside an envelope” – “is it used in an office?” – “was it used over 100 years ago?”
And the answer it arrives at is usually correct.
So how does it work? How did they program its data base?
One web site simply said that it uses a “neural network”. Can somebody explain this to me? I can see how a computer,(whether using a neural network, or just a fast binary search) can search all the possible permutations of 20 yeses and nos. But how did they categorize all the words in its vocabulary?
The toy has answered “labrador dog” in the animal category, when I was only intending it to pick “dog”. And it answered “dumbbell” when I was thinking of a weight -lifting exercise machine.
Who typed the word “dumbell” into its electric brain?And how does a pocket size $20 toy know how to categorize the word dumbell?
While that game seems to use a more sophisticated algorithm, it’s easy to write a program that constructs a “tree” of questions leading to answers at the “leaves”. I remember writing one myself as a means of familiarising myself with the Pascal programming language. You train the program by thinking of random objects and traversing its tree of questions until it reaches an answer. If the answer is wrong, you are asked to provide a question that distinguishes between the wrong answer and the right one, and the program adds this question and the new answer to its tree. Initially the tree consists of nothing but one arbitrary answer, “cat” for example.
If you can get a lot of people to use the program, by putting it on a website for example, you can speed up the training process immensely.
I also found this page, although it doesn’t say much more than the Kevin Kelly link I just posted (although it mentions the game “Animals”).
So, to answer your questions more explictly, they didn’t really program the database, at least not in the sense of selecting the objects and categorizing the questions; rather, they just took the most likely answers and links from the online learning system and hardcoded them in silicon.
Their use of “neural network” is somewhat misleading. They are actually using “decision trees” which, while they appear to be intelligent, actually don’t do any thinking at all.
Here’s how it works. The first person comes along and picks a random word, let’s say, DOG. The program asks the person to enter a question that will identify DOG, so the person types “is it short and furry”. Later, someone enters the word CAT. The computer gets to the question “is it short and furry” and if the person answers YES, then the program asks “is it DOG?” The person answers no, and the program knows it has nothing for this branch, so it tells the user to ask a question to differentiate CAT from DOG. Now the program knows 2 things, CAT and DOG, and can differentiate the two in just 2 questions. The next person comes along, and enters the word “SOCKET WRENCH”. The first question in the tree is “is it short and furry?” When the person enters NO, the progam has once again found a branch that it doesn’t have an answer to. The person enters a question to identify SOCKET WRENCH. Now the computer can identify three things, in at most 2 questions (if not short and furry it’s a socket wrench, if it is short and furry it has to ask a question to differentiate cat and dog). Someone else comes along thinking “WRENCH” and they will likely end up with “SOCKET WRENCH”, just as the OP ended up with labrador dog instead of dog.
This keeps going for many many years, as people keep asking more questions and putting in more objects. Once it has been running for a while (say, 17 years, according to the link in the previous post) it has quite a list of common objects that people will try and use. If you want to make a toy though, you are limited with how much memory you have to store all of your questions and objects. So, you sort for the top 2,000 objects, and hard code the questions and objects into the toy (it can’t learn) and it will get the most common objects that people try to put into it.
Their use of the word “nueral net” to describe the tree and “synaptic connections” to describe the tree branches is just advertising hype and has nothing at all to do with real nueral networks.
Yeah, but their program offers more choices than just Yes or No. While a decision tree algorithm could cope with more than two choices, it would generate a lot of overlapping data and would require an incredible amount of training. I don’t get the impression that they have used such an algorithtm. They appear to have used something that accommodates a gradation of answers between Yes and No, and that doesn’t require every point on the scale to have been addressed previously. I’m basing this judgement on the relative speeds of response of their web application at difficult decision points.
It also has some level of tolerance for conflicting answers. Using engineer_comp_geek’s example, a person might be thinking of “dog”. But when the program asks the person “is it short”, the person glances over at his Great Dane and says “no”. In the online version, the program can still generally reach “dog”, and will point out to the user that his notion of “dog” is a little different than the common notion.
Lets say we have Dog, Cat, Elephant, Fish and bird. Now each of those has certain chareteristics associated with it like mammal, can bark, eats mice, has tusks etc. etc. The program basically has this big data base of chareterstics associated with different object. In our example asking if the object is a mammal will eliminate at least 2 of the 5 objects so the program will ask if the object is a mammal. In essence it looks in its database and picks the chareterstic that is different in as close to half the objects in the data base. That way it hopefully will pare down the possible objects by 1/2 each question. For 20 questions that should (theoretically) be able to allow you to differentiate between 1,048,576 objects.
It does require a lot of training. But, guess who is doing the training, we are by playing it online!
I did a project similiar to this. We needed to know how words related to each other, some words are used frequently together or near each other, others are never used together or in the same context. We downloaded all of Wikipedia and built a decisioning tree. The actual database of relationships was fairly small, less than 10 megabytes and very fast.
I think it operates similiar to what you describe here. The only difference is there are preset questions. You can link each question with all possible matching answers. Once you get it narrowed down enough it presents you with the most likely choice. Probably the one a kid would choose or that has been chosen something like 95% of the time on the web.
I notice if you think of something really obscure it will ask you at the end to identify the object you are thinking of. The program is getting “smarter” by us just playing it. Going through a list of dictionary words and programming the characteristics of each one would be insanely hard and tedious. Even if you were to split everything up into groups like “mammals”, “can’t fly”, “has a tail”, “has tusks”, etc. You can’t get people to do that for free. But, we will play their silly game for free and do their work by answering their questions.