538 takes on another pollster

Less ‘takes on’ than ‘forced into conflict with’, I suppose.

Like Strategic Vision last year this time Research2000 is in trouble as analysts not involved with 538 noticed statistical problems with R2K’s results published at DailyKOS and shared them with Nate Silver over at 538. Now KOS has dropped R2K and is planning to sue and 538 is getting cease and desist letters from R2K attorneys. It’s a mess.

But kudos to Silver for not backing down. And kudos to all involved in polling and sampling for taking the industry to the next level: a level defined by self-analysis and evaluation that can force out the pretenders or the dishonest. Or at least reveal them to be dishonest for the world.

It’s nice to see a real world application of Benford’s law and its brethren.
Too bad about the shitty ‘pollster’ though.

I, for one, am not in love with the fact that we live in a world that makes a Nate Silver possible, not because I dislike the man. On the contrary, I admire him greatly. I just find it off-putting that someone, anyone, can predict the outcomes of events with the percentage of accuracy he has many times exhibited. Nate’s as close to a real life Hari Seldon as I’ve ever heard, and I’m just not that thrilled with the reality of being a data point in a math equation.

What’s next, the Mule? :wink:

Isn’t Research2000 the firm that did the poll that proved Republican voters were politically ignorant?

Is there a simple summary of the goings on out there?

Or is this a fair description:
From what I gather from the handful of stories and postings, DailyKos contracted Research2000 to do some polling for it. Other people notices statistical problems with the poll results (patterns that should have appeared did not), which strongly suggested that rather than actually polling people, Research2000 was making up results. Kos publicly fired Research2000 (giving credence to the allegations), and Nate Silver at 538 has been providing analysis and commentary about the supposed fraud. Research2000 has (or is about to) brought action against Kos and 538 (for various torts?) and is now suggesting the possibility of criminal charges.

What kind of polling results was Research2000 providing? Were they politically-minded results (e.g., happy puppy liberal isn’t Obama great! or look how evil real Americans think Obama is!)? That is, was Research2000 acting as sort of a push-poll rather than just taking the easy way out and fudging the results?

I thought they all did that?

R2K did Kos’ tracking pollings until recently:
Research 2000: Problems in plain sight

More on Research 2000

True, as I just looked it up, but a red herring, my friend. Those polls can be had for either side, for any time. And you know that. You disappoint me that you go for that sort of play.

The story here is that the guys involved in the analysis here (which is not Silver at 538) have again identified, just through the output, polling that looks fishy and have managed to place a firm against a wall with nothing but the math.

With any luck at all a few more cases of this (Strategic Vision, R2K, and others) will lead to greater openness in sourcing, questions, methodology and such that leads to better, non-bias polling. At a minimum it will put the firms that specialize in biased polling into greater relief and place those who use them under a spotlight.

Greater transparency and greater disclosure is NEVER a bad thing. That’s my reporter’s soul speaking. But I think I’m right.

Here’s a bit of rebutal for Bricker using data from other pollsters:
R2K or not, Republicans are still nuts

It’d be nice to have Rasmussen given a thorough look-see.

And your point is??

If there’s a lesson to be drawn from this example you bring up, it’s that Markos Moulitsas doesn’t want a pollster who’s simply going to tell him good stuff about his side and bad stuff about the other side - that his career of liberal political activism doesn’t override his desire for the truth.

Is there any information available as to which poll, precisely, is being questioned?

Unlike his opposition, which is quite happy to lie, cheat, and steal as long as it benefits them or their masters, the truth be damned.

The modern Republican Party is the single greatest danger this Republic has faced since the Civil War (aka the War of Northern Aggression).

Remind me again: who shot first?

It’s all the polls they’ve done. In fact, that is the core data that exposed the (alleged) fraud. It was the consistency of results from poll to poll to poll that alerted the researchers to the possibility of fraud.

Here’s just one example of the analysis, not even the one(s) that triggered the investigation. The evidence of fraud keeps piling up.

P.S. Click on “There’s more” for the entire article.

The thing that surprises me most about both this and the strategic vision scandal is that they probably could have pulled it off if they just invested in a random number generator. Given half an hour I could come up with fake data for a poll that is way more believable than this. It also makes me wonder that if the people who got caught are such bunglers, whether there might be other more sophisticated fraudsters still out there.

Even a stopped watch is correct twice a day.

Kudos to Markos Moulitsas for taking quick action against R2000. I don’t like his politics, but he’s helping to demonstrate why the blogosphere is better than the mainstream media - it self-corrects. Bloggers fact-check each other. Along the way, the truth filters to the top. For that to work, bloggers have to act the way Markos did when they’re caught in error, even if through no fault of their own.

And more Kudos to Nate Silver at 538, who has earned the respect of people on both sides of the political fence because he’s repeatedly shown that he goes where the math takes him, regardless of his own political ideology.

Check out the link to Benford’s Law that Squink posted. Often numbers that occur in nature aren’t random numbers–for instance the first digit of a number might be much more likely to be 1 than 9, depending on how the number. So a naive RNG is better than just making up numbers, but it still might not provide a data set comparable to real world data.

Aye, and he’s also earned massive respect among fellow statisticians. Last week I was talking to a consultant who specializes in data analysis of fundraising campaigns, who’s published the leading book in the field. Half of our conversation was about Nate Silver and his work in political and sports metrics. The man is taken very seriously, and this incident shows why.

A world with maths?

I was just over at Research 2000’s web site. They have not updated their ‘news’ section since April. I do not know just what that means, but it cannot be good.

If we live in an age where at least some pollsters are found to be cheats, who can guess how many were cheating throughout the years? Sort of puts a different light on everything.