That high school poll was hilarious! I wonder how they will bluff now that they’ve been called out.
OK, I gotta ask. Why the hell would a close election make certain numbers appear more often than others?
Because you’d be getting a lot of results of 51-49, or 52-48, and not so many ending in, say, 5.
Does anyone have a link to Strategic Vision’s website? It beggars belief nowadays that any “polling company,” however secretive in its methods, would not have one, but I can’t seem to find it by googling.
(Found via their LinkedIn profile)
Hmmm. That site describes Strategic Visions as “a full-service public relations and public affairs agency.” Not a polling agency. PR firms and polling agencies have very different, and incompatible, missions.
They’re not incompatible. A PR firm has to have accurate polling in order to know what their clients’ problems are.
Clearly if SVLCC is being hired to do polling, fails to do it, and makes up the numbers it is defrauding the client, and criminal charges might ensue.
But what if the client is in on it? What if Joe Politcal Candidate wants poll results showing him ahead of another candidate in a Congressional race, and hires SVLCC to produce the desired numbers? Joe then has a press conference trumpeting his lead. The actual election in the future may or may not be affected, and regardless the election results probably can’t be directly tied to the bogus poll results. Is this fraud? Who is the victim – the other candidate? All the voters? Can any criminal charges be brought?
Then 1 and 2 would counterbalance the 8 and 9. They do not.They are very low.
That assumes just two choices on the poll, though. For example, with three choices, A could have 48%, B could have 26%, and C could have 26%. I’m not sure what the corresponding chart would look like though.
Have, not do. A wise PR firm would recognize its own client-driven agenda and contract out the polling; only way to get accurate, unbiased results.
GameHat, I must thank you – I was addicted to 538.com during the election, but in the subsequent months have lost touch with the site due to political exhaustion. This has brought back my love of Nate and his wonkery. Plus the story is just delicious to watch unfold.
Yesterday’s update was an open letter from Nate to SV owner David Johnson:
Too freakin’ hilarious. I guess Nate’s not especially worried about all that many call centers cropping up. (Or the lawsuit, for that matter!)
There are always undecided and usually some some small number of votes for third parties. So it it more likely that you will the main candidates be 46-45 than 51-49.
Not only did no one get a perfect score, no one got 8 out of 10 or 9 out of 10 either. I have trouble believing that.
The result of these posts is ,that the numbers should balance out. They do not.
I have been following this. It is time for the guys at SV to grab their passports and go on a long vacation. Avoid Switzerland.
I suspect that there’s less to this than meets the eye.
The most fundamental problem with Nate Silver’s analysis is that his analysis of the Quinnipiac polling data shows that they too are not randomly distributed. They are not nearly as non-random as the SV data, but if they’re non-random then unless they too are frauds it suggests that his test is flawed. Until you have a proper understanding of just why it is that this test does not generally produce random numbers, you can’t use an even more non-random outcome to show anything at all.
And here is the bizarre part. Silver, in partial acknowledgement of this, suggests that the Quinnipiac data conforms to Benford’s Law. He links to a Wikipedia article on the subject. Problem is that the Wiki article makes clear that Benford’s Law says the leading digits tend to be lower digits. (Per Wiki, the law is also known as the “first-digit law”.) Nate Silver’s test was with the trailing digit.
And the difference is fundamental. The basis for the law is that growth is exponential, and thus slower in absolute terms at lower levels. For example it takes a lot longer to get from a salary of $100K to $200K than it does from $200K to $300K (assuming that a raise is typically a % of salary) & so on, so there will tend to be more people between the former two numbers than the latter two at any time. This has application for leading digits but not for trailing digits. (And FTM, it has no application to polls at all, even for leading digits.)
It’s hard to understand how Mr. Silver could just casually assert a connection to Benford’s Law without even addressing the above. To my mind that calls Silver’s integrity into question, and I wonder if he is a one eyed king of the blind men, dazzling non-math people with a knowledge of statistics that exceeds theirs, but not subject to anyone looking to closely at what he does himself.
Re the Oklahoma test, I agree that it’s strange (though I think some people are overestimating the amount of interest and knowledge 5th graders have in this type of info). I would guess that SV cooked the poll by disqualifying any answers that were misspellings and the like. This way they got an answer their client wanted to get.
In general, pollsters can produce results favorable to their clients’ agendas by careful phrasing of questions, or selection of sample group. I would guess that pollsters who are primarily PR outfits do this more than average.
So that’s my best guess. That SV is not an outright fraud. But they are not a highly capable pollster either. They don’t want anyone looking too closely at what they are doing because they will be exposed as PR hacks. But they’re not completely making up the polls either.
I could be wrong. I’m curious as to how it turns out.
Fotheringay-Phipps, I read the Wikipedia article on Benford’s law and was baffled as to why Silver would mention it in connection to a trailing digit pattern. I tentatively reached the conclusion that Silver probably knew more about it than the author of the wiki article, but I may have been giving him too much credit. Certainly the reason for the leading digit pattern cannot possibly apply to trailing digits. I wonder if anyone has addressed this issue on 538.
But if he did he should have mentioned it, considering that he himself linked to the Wiki article.
I went back to 538 and did a couple of things. I searched through the replies to the article the mentioned Benford’s Law. A number of people mentioned it, and some even suggested that it didn’t apply, but no one that I could find objected to it for the reason raised by F-P above. I registered and posted a reply which said:
I’m curious to see if anyone responds.