Of the 18 zillion polls conducted before the US election, how many were accurate WRT the actual results? Was there no “Bradley Effect” or was it minor? Why was the poll during the primaries (where Obama was ahead by 10% in the poll, but lost to Hillary by 2%–was that NH?) so far off?
It is not clear that the Bradley Effect was real, even for Bradley. His opponent’s senior strategist wrote an editorial that was published by the Washington Post (freely available until 11/16/2008) about that campaign and why the “Bradley effect” is a flawed analysis.
If you define accurate the way the polling organizations do, that is, within their posted margin of error, just about all of them were accurate. States that were called to be big Obama wins went big for Obama. States that were called to be big for McCain went that way. States that were too close to call ended up really close.
What happened to Obama in New Hampshire was a late surge for Clinton. Most of the pre-election polls had Obama at 35-39%. He actually received 36.4% – low, but within the margin for error. Clinton had averaged about 30% in the polls, but zoomed up to 39% in the final vote. That suggests most of the undecided vote went toward Clinton, rather than away from Obama.
fivethirtyeight.com gave a popular vote prediction of 52.3% Obama to 46.1 for McCain. The actual vote at CNN at this point is 53-46. I call that pretty accurate.
They actually took a cross-section of all nonpartisan polls and did their own calculations to get that number. Taking more than one poll is probably a good idea, since it will use more datapoints and various polling biases would tend to even out. However, it’s too expensive for any one poll to ask that many people, so they just try to keep a reasonable margin of error.
It might be worth noting here that if a poll has a very large margin of error, it might be accurate, but not precise. By contrast, a poll with a margin of error of ±1% but which was off by 10% would be very precise, but not very accurate.
The polls did very well.
Here’s a list of final poll results:
US: Obama 52, McCain 44 (Harris-10/30-11/3)
US: Obama 49, McCain 44 (GWU 11/02-03)
US: Obama 52, McCain 46 (Rasmussen 11/01-03)
US: Obama 51, McCain 46 (Daily Kos 11/01-03)
US: Obama 52, McCain 43 (Marist-11/3)
US: Obama 54, McCain 43 (Zogby 11/01-03)
US: Obama 53, McCain 44 (ABCPost 10-30-11/2)
With results plus or minus 3 points, they all did very well. The Bradley effect has been debunked pretty thoroughly. Remember that the Bradley election was a generation ago. Attitudes have changed considerably since then. Proof? The polls called most of the later primaries quite accurately.
Why didn’t they call New Hampshire? Mostly because a lot of people changed their minds at the last minute. Polls are snapshots of the previous couple of days, not predictions of the future. Getting close is all that can be expected. And close is exactly what all the polls did.
I don’t think that this is possible for a poll.
Possibly for a pollster - in other words, a single poll results in one figure, which may later be shown to be incorrect, but the error can not be determined to be due to randomness or bias.
So, if I shoot at a target once, I can measure how far away from dead center I am, but I can’t determine whether this is due to inaccuracy or imprecision.
They were even more impressive on the individual states. Everything which the polls showed as leaning red or blue, went red or blue, and the only neutral state was-you guessed it-Missouri, which had the closest vote (eventually went red).
The polls were excellent. Over in this thread, I made my predictions for the election based entirely on polls (with a tiny bit of gut instinct thrown in). I made only one mistake - Indiana threw me by going blue.
Impressively good job, pollsters.
Read up on the margin of error.
Why?
One notable failure of the polls has been the Alaska Senate race between Stevens and Begich. Most polls had Begich going in with a 10 point lead and yet the current numbers indicate Stevens winning by one point. However, voter turnout in Alaska was also abnormally low, leading some to speculate about voting irregularities.
Because it provides exactly the information that you’re saying a poll doesn’t provide.
No, I’m saying that it’s impossible to distinguish systemic inaccuracies from random errors in a single poll.
Fivethirtyeight.com missed Indiana. But that’s still a pretty impressive prediction, and they also were right on the nose when they said Obama would win by 6 percentage points.
electoral-vote.com now has a scorecard for the predix from various pollsters in the electoral college. Looks like he did as well if not slightly better than polls predicted. Conclusion he offers is that if anyone ever brings up Bradley Effect in the future, say “Well it wasn’t a factor in Prez Election 2008”.
Missouri hasn’t been decided yet, given that the separation between McCain and Obama is only .2%. Of course, it’s an open question whether there’ll be a recount, since it won’t substantially affect the results.
Keep in mind that they’re not even close to finishing counting absentee and early votes, for some reason.
Seriously, how does it take them that long?
Here’s the link to the analysis of the Bradley effect done by electoral-vote.com mentioned by AHunter3 in a previous post. As he says, in swing states the polling results were very close to the actual vote, except for New Mexico and Nevada, where Obama did better than predicted (but still within the margin of error).
Here’s electoral-vote.com’s comparison of the polls the morning of the election with the result. They predicted the winner of every state correctly except for Indiana (with Missouri as a tie).
Only to the extent that it’s not possible to know anything at all from inductive logic. If I conduct a poll with a very large sample size, such that it ought to have a margin of error of only ±1%, and it comes back off by 10%, then I know with a very great deal of certainty that something’s wrong. Ten-sigma results just don’t occur.
Regardless of who wins this race, it’s a stunning upset from a poll perspective.