His state predictions have uncertainties attached to them that represent (I think) the 95% confidence interval. For example, Virginia was projected to go 50.7/48.7 Dem/Pub with an error of +/- 2.5%. If we assume the 0.6% Jill Stein/Gary Johnson/Roseanne Barr/etc. remains constant, any Obama result between 48.2% and 53.2% would be considered a “hit” for Silver. He gave Obama a 79% chance of carrying the state, but Romney could have won it while Silver’s model was right, and Obama could have won even if the model were wrong.
To check his accuracy, you’d have to go state by state, comparing the actual vote shares to Silver’s predicted shares. With a 95% confidence interval, I’d expect hi to hit on something like 46-49 of the 50 states. If he hits on all of them, he’s underestimating the confidence of his model; if he his on too few, he’s overestimating it. I’d guess he’s got a pretty good handle on it, though.
Yes, Nate Silver colluded with the other pollsters in order to depress turnout and defeat the candidate sober enough to get America back on track again. Only time will tell how disastrous this presidency will be and whether patriotic Americans will allow the greatest country in the world to lapse into Populism and Communism.
The one relief is that the bien pensants were incapable of suppressing true Americans from acceding to the House and the State legislatures.
Fair dos to you, OMG. Just last week, I was sharing a drink with Nate in the Copacabana Room at the Polo Lounge, and he was looking really despondent. “I just don’t seem to be able to get through to this one guy over on the SDMB,” he told me. “Maybe I’m on the wrong track with this stuff after all.”
All I could do was tell him to buck up (and buy him another drink). He saw the wisdom in my advice and stuck to his model.
But you ALMOST got him to waver! Better luck next time.
Disclaimer: None of that little anecdote actually happened.
I think what might be more uncanny than the electoral college picks is that he appears to have gotten the margin in the national popular vote almost exactly right–and he was predicting that margin way back on May 31.
It should be noted, too, that his model is not just an average of polls. First of all the polls are carefully weighted based on their past reliability, adjusted for “house effects” (like Rasmussen’s rightward tilt), and–earlier in the cycle–for economic factors as well.
I understand the reasoning here, but I don’t think it’s correct. I don’t think Silver is calling probabilities, rather, I think he’s calling results and using probabilities to express a level of confidence. What will determine if I’m correct here is whether, since the things he had more confidence in turned out right for the past two cycles, his percentages start being closer to the 100 and 0 end of things (in other words he starts expressing more confidence using his probabilities).
The fair judge will be going state by state, both at Senate and Presidential levels, and seeing how close he was, how often he fell outside his range, and how that compared to other aggregator models.
For example, just starting at the top of his state Presidential list, he called CO as O +2.5% +/- 3 and it is currently at 4. (RCP rolling average had called it 1.5) Not bad. PA he called 5.9 +/- 2.6 and it went 5. (RCP 3.8) Someone needs to crunch all the races and put it head to head against other more simple aggregation models. (And I am not ambitious enough to do it myself.) First blush looks like all his adjustments improved substantially on from the simple averages but I’d love to see a systemic analysis.
And yes he made it clear in his final discussions about the remaining percent left for Romney being based on a systemic state bias that this was more a confidence interval.
It will be interesting to see if this changes things.
Is Silver the Bill James or Billy Beane of political predictions? Punditry that defies honest, objective analysis has been exposed as a stack of flyblown bullshit. How long will it takes the media to admit it?
I am a huge Nate Silver fan but to be honest I don’t think the election is necessarily a huge vindication of his model or anything. I think he himself said that he would be given too much credit if he called it right and too much criticism if he called it wrong. It was blindingly obvious the way the election was going from the polls and I don’t think his model actually adds that much value right before the election. The credit really belongs to the high-quality pollsters who basically got it right.
Where I think he did add enormous value were his lucid and deeply informative posts. I lost count of the number of times I had a question about the election and I would a find a post waiting on his blog with a detailed data-driven answer. He is so far above the the typical political pundit he might practically belong to another species.
Incidentally as I mentioned before I think a clearer test of his model is the Montana senate race. The last few polls have Tester up narrowly but he thinks Rehberg has a 66% chance based on “state fundamentals”. Tester is up four points but the race hasn’t been called yet. I will be deeply impressly with the 538 model if Rehberg pulls out a win but if he doesn’t, it is a data point that the model isn’t all **that **great.