Well, ignorance fought, then. I appreciate the explanation. You can probably tell I haven’t taken that many stats courses …
A question, though. If a candidate’s lead in a particular poll is less than that poll’s MoE, what precisely does that mean? The published lead is the most likely result, but it could conceivably range within that MoE? I’m genuinely curious.
I do see your point about aggregating polls lessening the possible range of error. That makes sense (how likely would they all be off in similar ways?).
What is published is the actual result of a random sample. The reality of the underlying distribution of voters could *conceivably *be anything consistent with the sample (i.e., it’s possible, though incredibly unlikely, that there are only 400 or so Obama voters and tens of millions of Romney voters). But the margin of error tells you how often a random sample could come up with these results and yet be further from reality than the margin. The typical margin of error is calculated at 95% confidence, meaning one in twenty polls would show that confidence interval just by bad luck of a very (randomly) skewed sample while the reality was actually outside of it.
To put this in concrete terms, suppose I have a bag full of thousands of red and blue marbles. I don’t know how many of each there are. But I draw 400 and get exactly 200 red and 200 blue. The most likely distribution of the bag is 50% red and 50% blue. But, of course, I could have just gotten a skewed sample just by chance. In fact, there’s a small chance that there were only 200 blue, and thousands more red. And there’s a fairly good chance that the bag is actually 51/49. But equally good that it’s 49/51. And so the likelihood of the true distribution of the bag clusters around my sample result and drops off on both sides, with 49/51 being more likely than 48/52, and so on.
Right. Very unlikely for them all to be off as a matter of random chance in the same direction. However, they could all be off because of methodological reasons, which are not accounted for in the margin of error. The margin of error describes only the possibility of the result being off by chance, not because of bad question design, or percentage of cell phones called, or any number of other methodological choices.
Whenever you’re doing a ‘meta analysis’ across multiple studies, on of the things you want to look for is ‘common mode error’. If everyone is making the same mistake, then the fact that many polls agree with each other is not necessarily indicative of where the race truly stands.
I’m not saying that’s the case here, but it’s an important factor to remember. Common Mode Error is one of the more common ways that statistical analyses, and especial meta-analyses, fail.
In the case of the current election polls, one possible source of common-mode error is the assumed ideological breakdown of the population. That’s what Republicans keep claiming - that the polls are over-weighting Democrats and skewing the results. Another possible source of common-mode error would be telephone polls that don’t account for cell phone or skype users, which would tend to underweight younger people. Democrats have made this claim to explain discrepancies among a number of polls.
Then there’s common mode error that no one is factoring because no one knows it exists (Rumsfeld’s “unknown unknowns”). It’s this stuff that keeps analysts awake at night, wondering if there’s a hidden factor they’re missing. The more complex the system that you’re trying to sample/model, the more of these that may be lurking.
That’s why I don’t get too worked up about the polls today. Why get into a big argument about which polls are accurate and which ones aren’t? We’ll all know in nine days. Then we can have fun debates dissecting them and figuring out why the ones that were wrong failed.
We’ve done this one to death, but it’s always worth pointing out that this could not actually function as a systemic error in the current aggregates because only a handful of polls weight by party ID (most prominently Rasmussen, so that’s another reason it wouldn’t play a role).
A far more likely systemic error is assumptions about the demographics of the eventual electorate. If white turnout is 80% instead of 77% or so, then most of the polls will be underestimating Romney.
Exactly right. The polls that allegedly showed too much Party ID were notably NOT balancing for party ID; that’s just how random samples once corrected for demography fell out. But aye there’s the rub. There could be systemic errors in those demographic corrections, or in the likely voter screens.
2ManyTacos, we seem to have a habit of posting at the same time! Thanks for that Rand poll link. What an interesting methodology. Not so sure what to think of that approach though. Still it is of note to see another approach give a different sort of result.
No. I don’t know anyone who thinks, and have read no reports to the effect, that Portman is making a big difference for Romney here in Ohio. (Any kind of difference at all, actually). Romney will win or lose on his own merits, and those of the President.
I like this sentence best (emphasis & numbering mine):
“Our discussion repeatedly circled back to the nation’s single most important challenge: [1]pulling the economy out of the doldrums, [2]getting more Americans back in the workforce in meaningful jobs with promising futures, and [3]getting the federal government on a track to balance the budget in a bipartisan manner that the country demands.”
Must make those who highlighted the Salt Lake City paper feel stupid. The Des Moines Register endorsing Romney is by far the biggest deal when it comes to newspaper endorsements. They haven’t endorsed a REpublican since Nixon.
Nixon was running against McGovern, who while definitely the better man, was probably the worst Presidential candidate in the 20th century. I’d suspect there are a lot of papers normally Democratic that endorsed Nixon in 1972.
Anyway, Romney swept the Iowa paper endorsements, getting all four major dailies. Be interesting to see if that makes a difference in Iowa.
The Register picked Nixon? You sure that is the factoid you want to push? What should I conclude from that? That whenever the Register picks a Republican, vote for the other guy?
Exactly. If I were asked to provide one sign of Republicans’ having no sense of shame, just balls-out denial that they’ve been wrong when they’ve provably wrong, it would be their overwhelming joyous pride in Nixon’s victory over McGovern in 1972.
Oh my God. This is a microcosmic example of everything that’s wrong with our country.
No, the Des Moines Register could not have reasonably be expected in 1972 to realize that they were endorsing a guy who had committed the crimes that Nixon committed. There is nothing unreasonable at all about their endorsement.
No, that fact has zip-all to do with today’s political situation. People, we are seriously in here fighting about what deeper meaning the Des Moines fucking Register’s presidential endorsement has. Come on. There’s only like a week to go. Hold it together. We can do this.
Will it make Rob Portman look bad if Romney loses Ohio?