Are the Polls being Gamed this Year?

Politics & Elections
1

I was just looking at the RealClearPolitics poll summary page here: Presidential Primary Polls

The polls are just incredibly bad this year. Check these results from the last polls in the completed states:

Alabama:
Last Poll: Trump +23
Result: Trump +21

Georgia:
Last Poll: Trump +19
Result: Trump +14

Kentucky:
Last Poll: Trump +13
Result: Trump +4

Louisiana:
Last Polls: Trump +13, Trump +17
Result: Trump +4

Okahoma:
Last Poll: Trump +13
Result: Cruz +6

Texas:
Last Poll: Cruz +3
Result: Cruz +17

I’m seeing the same thing on the Democratic side, except the polls heavily overrepresented Clinton’s support. For example, in Oklahoma the last poll had Clinton up by +9, when in fact Bernie won the state by 10 points. That’s a huge error. The last Massachussetts poll had Clinton up by 9, and she barely squeaked out a win at +1.5.

On the Republican side, every single poll I looked at was wrong in the same direction: They’re all significantly over-stating the support for Donald Trump. On the Democratic side, most of the polls had too much support for Hillary, but with a couple that went the other way.

In any event, these are not small errors. In some cases we’re talking about 10 point differences - in Oklahoma, a 19 point difference between polling and actual results. That makes the polls there not much better than throwing a dart at a dartboard.

So what’s going on? I don’t buy that it’s just cell phones or some other methodological flaw in the internet era - we’ve got plenty of other polling on other issues that don’t show this huge gap between the polls and reality, and it’s not clear how these issues would always break for the same candidate.

My suspicion is that lots of Democrats are claiming to be Republicans just so they can pump up Trump’s numbers and help him ruin the GOP nomination. But it could also be that Trump’s name recognition is causing people to pick him in casual polling, but when they actually have to pull the lever for him they just can’t do it.

In any event, the polls are really, really wrong this year.

2

There are a lot more organizations doing polling, using different methods. Caucus states are notoriously difficult to poll. Plus, some of these states just don’t get polled often and even experienced pollsters have difficulty polling democrats in Oklahoma for example.

3

I can’t explain all of it, but polling has always been terrible in caucus states. A caucus demands more commitment from the voter than a primary where you can walk in any time, drop in your vote, and leave. The higher the level of commitment you ask for, the more people are going to claim they’re a likely voter and then not go through with it.

Most of Trump’s support is coming from people who usually are less likely to vote. If they are claiming to be more likely voters this time around, but not showing up as often as they claim, that could explain the Donald’s underepresentation at the voting booth. Scary, he’s still winning. According to polls, it should be even worse.

Similarly, Clinton’s numbers are very high among African-Americans. They are probably also underepresented in the voting booth.

4

The OP concludes that there’s something different about the polls this year, but only talks about this year’s polls.

Did the polls do better the last time Oklahoma had a competitive primary race?

5

Some polls are just really really bad.

If the RCP average rolls in some of those bad polls…

6

Do you have any examples of these other current super accurate polls? And how do you tell if a poll is accurate or not if there isn’t something like an election to test the results?

I’d certainly say that the decline of landlines, falling response rate, and the greater difficulty in reaching a representative demographic are all increasing challenges to pollsters. It’s entirely plausible that these issues could lead to overrepresentation for a single candidate (eg Trump supporters are really angry and more likely to answer a pollster or survey in order to spout off, or Sanders supporters are younger and less likely to answer telephone polls).

Here’s a decent article on some of the issues: Opinion | What’s the Matter With Polling? - The New York Times

7

(1) Caucuses are hard to poll well
(2) State primary elections tend to get polled by smaller state outfits which often aren’t as good
(3) State primary elections tend to get polled less, so bad polls impact the average harder.
(4) Late deciders have been breaking far and away against Trump. So a poll with Trump at +7 but 12% undecided can easily whittle his lead down to nearly nothing.

8

Why? Don’t they know her phone number?

9

Sam, you are trying to make a statistical argument but you are cherry picking data points to make it.

Here are the differences between polling and actual results for states with enough data that RCP could calculate an average. All numbers are in terms of Clinton’s margin.


		RCP	Actual	Diff
Iowa		4	0.2	-3.8
New Hampshire	-13.3	-22.4	-9.1
Nevada		2.4	5.5	3.1
South Carolina	27.5	47.5	20
Arkansas	28.5	36.6	8.1
Georgia		37	42.9	5.9
Massachusetts	6.7	1.4	-5.3
Oklahoma	2	-10.4	-12.4
Tennessee	26	33.7	7.7
Texas		30.3	32	1.7
Vermont		-75	-72.5	2.5
Virginia	21.5	29.1	7.6
Louisiana	39	47.9	8.9

In only four of thirteen did the polling average overrepresent Clinton’s support, while they did overrepresented Sanders in nine out thirteen. The largest error in Clinton’s favor was indeed in Oklahoma as you pointed out and it was in the Clinton direction. However, the biggest miss overall was in South Carolina and it was in the Sanders direction.

You are not seeing the same thing on the Democratic side.

10

Yeah, i would expect Bernie to underperform his poll numbers, same as Trump. Both of them appeal to non voters.

11

I didn’t look at the Democratic side as carefully, to be honest. I did say that it wasn’t as clear on that side, and noted polls were wrong both ways on that side, although it looked more Clinton skewed than I guess it was.

12

Yeah, on the Dem sides it’s mostly been margin of victories for the respective candidates that the polls have missed. States where Clinton has won big, the polls have tended to underestimate just how big she’d win, and likewise for the States Sanders has won (though he hasn’t won nearly as many with huge margins like Hillary, the ones he’s won he’s tended to win by more than he was expected to win.)

13

ISTM that exit polls have consistently shown late-deciding voters breaking against Trump. So it’s not that the polls were wrong, but that things changed after they were conducted.

[As for why the late breakers tend to go against Trump, my guess is that Trump-as-a-concept is more appealing than the actual Trump himself, so when people pay closer attention his support declines.]

14

My point is that it is not actually skewed Clinton at all. On average it is skewed Sanders. You said in the OP , “…the polls heavily overrepresented Clinton’s support.” The opposite of that is true.

I haven’t looked at the republican side yet, but I don’t see the point explaining the skew in the polls until you demonstrate that there is one.

15

I think a lot of these polls also have large undecided percentages that sometimes aren’t reported in the aggregation sites, which can explain some of it as well.

16

So, out of boredom, I have compiled a table of how Trump performed compared to his RCP average in primaries and caucuses where there was enough info for RCP to post an average.


		RCP	Actual	Diff
Iowa		28.6	24.3	-4.3
New Hampshire	31.2	35.3	4.1
South Carolina	31.8	32.5	0.7
Alabama		38	43.4	5.4
Georgia		36.2	38.8	2.6
Massachusetts	45.3	49.3	4.0
Oklahoma	32.7	28.3	-4.4
Texas		28.2	26.7	-1.5
Virginia	36.8	34.7	-2.1
Louisiana	43.3	41.4	-1.9

Trump underperformed five times and overperformed five times. Sam, are you still suspicious that, “that lots of Democrats are claiming to be Republicans just so they can pump up Trump’s numbers”?

17

One possibility is that Trump is appealing to people who feel the existing political system is broken. People who feel that way are unlikely to be regular primary voters. So it’s possible that a lot of people are verbally offering support to Trump but aren’t following through by actually registering and voting in a primary.

18

I’ve also been reading that the voters who decide at the last minute are heavily against Trump. Add that to the general unreliability of primary polls and the lack of pinpoint prediction is a given.

In general, my advice here has always been to pay minimal attention to primary polls in general and zero attention to any one individual poll ever. Trend lines over many polls and months may have some predictive ability, however.

Maybe we could make this a sticky:

THE POLLS WILL BE ALL OVER THE PLACE FOR THE NEXT EIGHT MONTHS. PLEASE DON’T GO BLATHERING ON ABOUT THEM AS IF THEY ARE MEANINGFUL.

Yeah, that’s gonna work.