Near the start of the article is an illustration of two contestants and a link to play a game against Watson. So put your thinking caps on, and limber up your fingers. Please note that the questions are the same for all players, so if you want to discuss specific answers, please use a spoiler box.
Yeah, I was surprised by how many questions he didn’t have the right answer to. Not sure what was up with that. I admit, however, I didn’t read the entire article; possibly it addressed his shortcomings.
I scored 48-15. I think the simulation is dumbed down to spare the egos of the folks at home. It was informative… it shows there is a confidence threshold in the algorithm.
Interestingly, the “thinking” sometimes matched my own. I would be unsure whether to risk answering or to just “pass” instead. Also, sometimes the possibilities I was considering matched Watson’s fairly closely.
I’m glad the parser is fairly merciful in acceptable answers. It is certainly not as strict as Trebek would be.
I really hope the real Watson is better than that…I kicked its ass (40 to 16, I got all but 8 answers on the board, 2 of them Watson didn’t get either - and one of the ones we both missed, it was because I mistyped).
I started out with having him as low as -3 after three questions, but then blew a few easy ones (including one mistype) and ended up winning only 29-18.
Yes, a friend from another board took the test a second time, passing on every question. He felt that the results weren’t very impressive. He reported that it got seventeen correct answers, and only two incorrect, but thought that it didn’t do well on the other eleven questions. When Watson’s answer is “I’m not sure”, it gives the responses it “considered”, in order of probability – a guess, essentially. Watson’s guesswork would have yielded a correct answer only four times out of eleven.
He also decided that unless Watson is “playing the digital equivalent of possum”, it has problems with context & category. And he noted that most of the questions in this quiz were fairly easy by “Jeopardy!” standards – first-round stuff, not real challengers.
I wouldn’t be a bit surprised if Watson had been “dumbed down” a bit for this game. How much fun would it be if everyone who played got blown away by Watson? Although, without of the time pressure of who can ring in first, it’s much easier for the human contestant to get a decent score.
I agree. I found it very easy to defeat Watson, though on some of the questions I found more difficult, it took me a few moments of thinking. I doubt I would have done nearly so well had speed of recall been a major part of the game.
Keep in mind that the article is from June of last year. Even if the answers in the game came from the real Watson, it won’t be the same Watson that is going up against Ken and Brad. IBM has had time to improve it.
In page six of the article, the writer believes that Watson at that time wasn’t good enough to go against Ken. It was beaten quite a few times by people who aren’t anywhere near as good as Ken, including people that didn’t come in first on their appearances on the show.
Have they improved it enough to offer a real challenge? Guess we’ll find out in a little over a day.