Various outfits have been producing scorecards of pollsters lately. Poblano at Five Thirty-Eight has a good one, but it’s strictly about general elections, so its applicability to the present primary season is iffy.
SurveyUSA has a report card of the 2008 primaries through Wisconsin - actually two: one ranked by median differences from the ultimate results, and the other ranked by mean differences. It’s pretty useful, though it’s got some problems that are too complicated for this post.
One meme that’s been bouncing around the blogosphere is the claim that some pollsters whose overall track record is iffy, are adapting to polling under the special circumstances of this campaign, and some pollsters whose overall track record is good, are starting to look less brilliant than they were earlier.
So I decided to do a pollster scorecard of just OH, TX, and PA, since a lot of pollsters had polled all three. (I took a pollster’s most recent pre-primary poll, but excluded anyone who hadn’t polled within the last week before the primary.) I used whole numbers for the margins, except for TX where Clinton won the popular vote by 3.5%, and ISTM that rounding, either way, would advantage the pollsters who erred in the direction I rounded. So half-points for Texas.
Here’s the six pollsters who showed up in all three states, and how far off they were from the actual margin of victory in each:
State OH TX PA
SurveyUSA 0 4.5 3
Zogby 10 0.5 1
ARG 4 0.5 7
Rasmussen 4 4.5 4
Mason-Dixon 6 4.5 4
PPP 1 2.5 12
Maybe not so much. SUSA’s still best, and while Zogby and ARG are doing OK overall, Zogby’s got one serious clunker (calling Ohio a tie), and ARG’s got one middlin’ clunker (calling PA for Clinton by 16). Whether they’re better than Rasmussen depends on how you rate a mixture of being on the nose and way off, versus a string of middlin’ misses such as Rasmussen had. I personally prefer a pollster that never exactly nails it, but never totally blows it, to one that sometimes nails it, and sometimes seems to be in the wrong universe: at least I don’t have to worry about which it is, this time. But YMMV.
From a practical standpoint, one would turn my bias into a rating by summing the squares (or some other power) of the errors.
No matter how you cut it, SUSA still looks awfully good, PPP’s bad day in PA kills them, and ARG, as much as I hate to say it, fights its way back to semi-respectability. RCP’s been asterisking them and not including them in their averages, but I’d say it’s time for RCP to reconsider that.
Pollsters that surveyed just two of the three states did quite well. Insider Advantage has been polling NC, and given how they did in TX and PA, you’ve gotta take them seriously. (Currently they have Obama up by 5 in NC.)
State OH TX PA
Suffolk 2 --- 1
Insider Advantage - 1.5 2
Quinnipiac 6 --- 2