it has them rattled.
a switch is thrown turning on a light when the answer is completed. that switch also signals Watson.
One hopes not, as he was programmed to play a typical game of Jeopardy (or as typical as possible with his lack of sensory apparati), and clues occasionally do include at least part of the correct answer - there’s even a whole recurring category based on it (Stupid Answers).
I’m a huge track and field geek, and I think that Watson was essentially false starting on all of those questions. After Watson ran about 15 questions, I turned it to something else.
In track, the sprinters are in the blocks and they receive the signal to start which is the starting gun. They have to process the sound, tell the muscles to move, and start moving. There are sensors in the starting blocks, and if the sprinter reacts to the gun in less than 0.100 seconds it is deemed a false start. The difference is calculated by sensors in the starting gun and in the starting blocks. That difference was determined through some research about how fast a human can physically react to a stimulus.
I’ve looked over lots of data from world championship and Olympic track meets, and rarely do sprinters react faster than 0.120 seconds. Sprinters do occasionally react at less than 0.110, but a quick statistical look at all reaction times clearly indicates that it is an outlier, and the sprinter had to have anticipated the gun. And gotten lucky.
Watson, though, can react to the stimulus in much less than 0.100 seconds since it is dealing with circuitry that is much, much faster than human reflexes. So the only hope that the humans had was to anticipate the end of the question and still somehow get in under the time that Watson takes to respond. I’m guessing that that is a window of less than 0.050 seconds. So if the human correctly anticipates the end of the question in a window of about 0.050 seconds, he would beat Watson to the buzzer. Otherwise, no.
There are also occasionally multiple choice clues. I remember a category called “Not a Current National Capital.” Each clue listed three cities and the correct response was one of the three. I would be interested to see how well Watson would do in a category like that. Would he understand the “not” immediately? If not, would he learn how to play the category after seeing what a couple of correct responses are?
Also, I was disappointed that there wasn’t a wordplay category. One category yesterday involved words with alternate meanings, but that’s pretty easy. I’d like to see Watson try a Rhyme Time category, or a “before and after” or something involving puns, or anagrams, or “5-syllable words.” For the more straightforward trivia questions, it really doesn’t surprise me at all that the computer can beat the humans.
I’m still waiting for John Connor to show up and blow Watson into a thousand pieces.
Jeopardy has always been about reaction time as well as knowledge. Some smart people probably do badly due to slow reflexes. It’s a given that a computer has a super fast reaction time.
Yeah, I have to think that they need to do something about the button mashing issue. The humans just got shut out of something like 20 questions in a row and several were pretty easy, so the computer must have just timed the button press perfectly.
The computer was very, very impressive on the question answering, though.
I, for one, call BS on Watson being fed the questions by text. It’s not a real contest if he’s not using speech recognition.
I also call BS on the timing of buzzing in.
What I thought would be interesting (and it was way more than I thought) was how he would wager on Daily Doubles and Final Jeopardy. Since he would have extra time to process an answer, and since he gets 90+% correct, it would be logical to have him bet the max on DD and trivial to figure out the most to bet on FJ that wouldn’t cause him to loose. However, they apparently programmed him with an unnecessarily complicated formula… or a random amount. Since he was obviously programmed to win, I am really curious as to how he came up with the wagers that he did.
I doubt it was programmed that way.
How could Watson think Toronto is part of the US? I guess it could be some major bug that caused that. Or maybe it was fed some bad data.
Why is speech recognition needed? The humans don’t have to listen to Trebek, they can just read the clue. A deaf person could play the game if they could talk.
He probably (incorrectly) parsed the ‘US’ part as unimportant - remember the big issue with natural language inputs is that computers have a really hard time figuring out what’s important and what’s not.
There were only 2 words in the category name - US and Cities. How could US not be important?
If the category was black musicians then black would be important.
I could see the software telling him if you are way ahead in final jeopardy , bet a low amount and if you are not sure of the answer come up with something odd like Toronto.
So, if it’s cheating for the computer to use its own reflexes, why isn’t it cheating for the humans to do the same? Whoever has the best reflexes has an advantage: That’s part of the game. Yeah, Watson has strengths that humans don’t have, but then again, he also has weaknesses humans don’t have. Should the human contestants not be allowed to hear other contestants’ wrong answers, since Watson can’t?
He guessed his answer based on the clue, not the category.
Also, they’ve spent a lot of time prepping Watson to get puns and, specifically, to play Before And After. Those are obviously some of his known weaknesses, but he wouldn’t get the floor wiped with him.
Is Watson processing the right ‘answers’ that it did not get right? It knew (after a pow-wow) that his answer of leg for the gymnast question was wrong. But does he know why it was wrong? and what the right answer was? and how is that information being transmitted to it?
I believe he is - it might not be happening during the game (though I would assume it would be), but he’s already gone through a learning process, so I assume he’s capable of upgrading his knowledge/logic base, and unless it has to be entered by a human handler, I can’t imagine why they’d turn it off for the games, and even if it does, I can’t see any good reason not to do at least a cursory ‘remember to read the category, dude’ run between rounds.
I’m betting some programmer gets fired over “Toronto”. It had 30 seconds! Once you understand the question, the search algorithm is easy as hell - run a list of airport names against major WWII battles (since that’s easier than people’s names), then check the resulting list against the list of cities with 2 airports. Clearly it was spinning its wheels and had no idea what the clue meant.
They have a guy who works on the wordplay questions.
John Prager
John Prager works in question analysis and categorization, developing algorithms for special question processing that handle such things as wordplay
Well, that’s the crux, ain’t it?
ETA: Still, I’m really curious how it could end up with the answer Toronto. Seems like a simple keyword search word point strongly to the right answer. Did they show the other possibilities and confidence percentages for this one?
At first I thought of NYC since they have 2 airports but then I realized Chicago had Midway although I did not know that was named after the WW II battle.
Sure, but it seems like if it could parse the other clues, it should have been able to parse this one. Data points:
US City
2 airports
Named after WWII hero
Named after WWII battle
Once you’ve got that, with 30 seconds to check databases, and given that the answer is one of the largest cities in the US, it should have been easy. Clearly it couldn’t parse that clue, which is odd considering some of the apparently trickier ones it parsed earlier. Maybe it was just too many data points for it to separate. Heck, if it had answered any other US city with 2 airports , I wouldn’t be nearly as surprised.