It started 9.24 - the American Association for Public Opinion Research reprimanded SVLLC for “failing to disclose even basic information about their political polls” (quote from Silver’s blog)
9.26 - Silver responds to the most common complaint on his message boards - that he doesn’t apply the statistics indicating fraud on the part of SVLLC to other pollsters. Here he compares Quinnipac to SVLLC. SVLLC tells a reporter that they plan to sue him.
9.26 - Silver dissects an SVLLC poll that claims Oklahoma high school students lack basic political knowledge
Things are still developing, but I’m glued to the computer. I read 538 religiously during the last election cycle and was extremely impressed at how good Silver was at breaking down the math of the many different political polls and the ramifications of such.
A year ago, when told that he was originally a baseball statistician, I realized that Nate Silver was incorruptible. I’ve spent my life with the type, and there is NOTHING more important to them than the Numbers.
After that 538 became as close to gospel as could be found online.
This is the first I’ve heard of it. However, the first link does acknowledge that SVLCC is not a member of the organization. There for it has no obligation whatsoever to provide any information at all to the national membership organization.
If SVLCC is cooking the numbers, I suppose the next reasonable question is why. It could be they are defrauding their own clients, which becomes a case for state criminal justice departments. But it could also be that they are acting as a cover for their clients – that they simply produce the numbers their clients want – and the clients use them however they do. I’m not even sure that’s a crime at all.
Then again SVLCC could simply be a very paranoid organization that considers all their methology to be proprietary, and simply won’t provide it as a matter of policy.
It’s not just that SVLLC is refusing to disclose it’s polling methodologies. It’s that their published data shows disturbingly non-random results that other polls would be expected to show.
As for why, it’s not hard to deduce. It’s a lot cheaper to make up some numbers, then sell them “polling results” than it is to actually do the groundwork to collect such numbers.
Looking at his math, I’d say he’s got a pretty solid case as well. However, the caveat is the one he himself mentions - perhaps there’s something about the nature of the polls they do that is causing the numerical bias. For example, if they are only hired in cases where the election is very close, you might expect certain numbers to appear more often than others.
The other problem with the analysis (which he also mentions) is that even the ‘control’ he used shows significant non-randomness. So it’s not a complete slam-dunk. But it is strong evidence that something’s afoot. As is Strategic Vision’s refusal to release their methodology and supporting data.
Yeah, he’s built a pretty good circumstantial case. And it’s likely impacted their business already. It’ll be up to the real journalists to finish the job.
Meanwhile, over on pollster.com, Mark Blumenthal has some more questions. Also, some concerns in general about pollster transparency that this issue brings to light. And given that I found this linked from Andrew Sullivan’s blog, it looks like the story is picking up some momentum.
While I think some of the other posts are more damning in actually proving something’s up, the Oklahoma high school polling post looks the worst on its face.
1000 students were asked an no one got a perfect score? And only 28 got 6 questions or more correct? Only 230 knew who the first president was, and 270 knew the parts of Congress? Even if OK was the concentration of Stupid in the northern hemisphere, there has to be at least one smart kid making sure the others don’t drown in their own drool.
As an aside, 110 students thought it was the Republican Party and the Communist Party. I’d like to pretend an equal number said the Democratic Party and the Fascist Party, or the Cute Furry Animal Party and the Puppy Eater/Seal Clubbing Party. If you’re going to lie about your numbers to support your bias, subtlety is your friend!!!
Question: Do political polling firms release their data sets? I would expect that Quinnipiac would, being as they’re an academic institution, but do Gallup, Rasmussen, et al.?
I wouldn’t go against Nate Silver when it come to crunching numbers. I hope the “polling company” in question does take him to court, because they will have their corporate ass handed to them if they do.