Why is Gallup oversampling Republicans? (Or are they?)

Great Debates
1

Moveon.org has an ad out today (pdf) accusing Gallup of heavily oversampling Republicans in its election polls. According to the ad, Gallup’s samples of likely voters have Republican turnout exceeding the Democrats by 6-8 points. In fact, in the last two elections, Democrat turnout has exceeded Republican by 4-5 points.

If my understanding is correct, this is a deliberate choice made by Gallup. They have to put together a sample that looks as much as possible like the turnout on election day. So why do they think such a big change is going to occur this year?

2

According to the Columbia Journalism Review:

That article doesn’t directly address whether Gallup had oversample Republicans or not, but the CJR does have a couple of other articles on polling, biases, and news organizations:
http://www.campaigndesk.org/archives/000925.asp
http://www.campaigndesk.org/archives/000922.asp

3

One thing I’ve heard is that most telephone opinion polls do not call cell phone numbers. Cell phone owners tend to be disproportionately Democrats, so excluding them skews the results.

4

Try this:

5

I find this bizarre. Why would this be?

6

Part of it is this which was quoted earlier:

“Democratic strategists say that the national surveys underrepresent cell phone users, who tend to be younger, more likely to live in cities and more open to Kerry’s message.”

The part about cities is technical: low population density in rural areas mean there is often no cell phone coverage; or poor coverage. As for being younger, in your experience are cell phones that popular with the elderly compared to the young? Younger people typically are more accepting of newfangled technology. Older people tend to be more conservative; and also wealthier than young adults. One other thing that occurs to me is, at least in my experience, cell phones tend to be very popular amongst urban minorities. I can’t swing a dead cat in the hood around here without hitting someone with a cell phone. (Although, if I did that the likely reaction would be “Why is that stupid honky swinging a dead cat in this part of town?” :wink:

7

Hmm, are they oversampling Republicans or not?

Well, the latest Gallup poll has Dubya by 13 points. Washington Post/ABC has him up by seven. CBS has him up by eight. Time has him up by four. Fox has him up by two, within the margin of error and thus a statistical tie. The Marist poll is the same as Fox. Investor’s Business Daily/Christian Science Monitor has Kerry by one among likely voters, tied among registered voters.

Let’s play the old Sesame Street game, “One of these polls is not like the others/One of these polls just doesn’t belong . . .” The Gallup poll is an outlier. It can be tossed, if you want a clear picture of where things stand right now.

The issue here is that Gallup has been doing this on a regular basis. According to Polling Report, Bush had a four-point lead DURING THE DEMOCRATIC CONVENTION at the end of July (likely voters). Since then it’s only grown. This strongly indicates that their methodology is suspect, and that they are oversampling Republicans.

Here’s another piece of evidence. The Gallup poll wouldn’t be oversampling Republicans if, in fact, the electorate was turning heavily Republican. But does party ID shift all that much? No, according to this study. Scroll down a little bit to the first gray table. It shows a party ID survey by Pew Research done over the years; the 2004 survey included over 19,000 people, so its MOE is small. It consistently shows more people identifying as Democrat than Republican, going back to 1987, and only in 1995 does the gap favor the GOP. The 2004 figures, taken from the first six months of the year, show a four-point gap in favor of Democrats.

Further down the page the author does a striking comparision of six recent polls that weight by party ID, versus six polls that don’t (including Gallup). Of the polls that weight, the author notes: “The similarity between the results in these six polls is remarkable. The race varies from Bush up four to Kerry up one, with no two polls disagreeing about Bush’s raw score by more than three points or Kerry’s raw score by more than four points. On average, Bush leads by less than two points (47.3-45.5).” In other words, polls weighted by party ID show the race to be close, essentially tied.

He goes on to note that polls that do not weight by party ID show much more movement within themselves at different time periods, than polls that do.

Finally, he “fixes” the unweighted polls by weighing them by party ID (table 5). They look almost exactly like the weighted polls.
Frankly, Gallup looks like trash. I know why they’re doing what they’re doing; they want Dubya to head into Election Day with an aura of invincibility.. What I haven’t heard is the backstory as to why Gallup, Jr, became such a gung-ho GOPpy after his father died. Anyone?

8

Oh, and as for cell-phone users? It’s my understanding that the issue here concerns people who use cell phones exclusively --i.e., they don’t have a land line at their home. These people are passed over for polling and as one may expect, strongly tend to be younger, live in urban areas, and hence skew Democratic.

9

That would be another issue, but I posted of others above. These people who use cell phones exclusively are a significant reason why they are popular amongst poorer minorities. For many, using a cell phone exclusively is cheaper than getting a landline phone. And, you have phone access when away from home.

10

The cell phone bit is interesting, but it really is a sidebar to the main issue: Why does Gallup seem to think that so many more Republicans are going to vote this year? (Or, at least, what is their stated rationale?)

11

You guys should read this article from the Pew Research Center (who I’m familiar with, but am not sure if they’re generally considered biased).

Basically, it says that the party affiliation in polls reflects the current attitude of the voter as much as it reflects their actual registered status:

Party identification is not a particularly stable attitude[,] many people go back and forth on how they regard themselves. When respondents are surveyed and then re-interviewed at a later date, substantial minorities give different answers.

It goes on to say that their own polls have shown that up to 16% of people will report a different party identification over a mere three weeks. So party identification can be pretty fluid over the short term.

12

Whoops, The bolded part above is a quote from the article. I’m no plagiarist!

13

One possibility: sampling error. This is where the cell phone bit comes in. If they are calling landline phones, and Democrats tend to use cell phones more often…

14

But don’t they assemble the sample itself based on this assumption? They don’t just call until they have 1000 likely voters. They call until they can put together a sample of likely voters that is x% Democrats, y% Republicans, and z% Independents. Once they get what they have defined as their quota for, say, Democrats, they stop polling them.

Am I wrong?

15

[hijack]I don’t have a good answer as to why they’d oversample Republicans, except that they believe Republicans are more likely to vote (which the OP says hasn’t been true).

But I don’t think saying Bush has a bigger lead than he does makes an organization biased. If I were a polling place that wanted to make my person win, I’d say he was losing by a slim margin, so everybody who’s for my guy would be more motivated to get out and vote.

But then, I’m a Democrat. That’s how Democrats win elections. Maybe they really are just trying to discourage voters, making it look hopeless.

16

No, the idea is to create an “aura of inevitability” around your candidate. One way to do that is to manipulate poll numbers to make it look that way.

The folklore is that we Americans love underdogs, but ISTM we love winners better. Don’t we all want to be on the winning side?

Are they doing it? This link says so, FWIW:

That makes an 8-point Bush lead, after a 12-point “adjustment” in his favor.

17

There’s a science to it. Right now the candidates are still fighting for financial contributions and endorsements; these are likely to be held back if a candidate looks like he can’t win. In the last few days, the candidates want to motivate voters to go to the polls (or not if those voters support your opponent); this is when you want it to look like the race is neck and neck.

18

I assume the majority of polling is done via telephone.

Is there any bias as to who is more likely to be at home to receive these calls?

I’d think it would be more in a Conservative’s nature to be home during the evening. Family oriented, home owner, doesn’t go out much.
As where a Liberal may not be apt to be home at night. Singles, out dining, traveling, renters.

19

Yup. Somebody with a lot of money may be, as well as more likely to support the tax-cutting Republicans, more likely not to have to work a second or third job in the evening just to pay the rent, let alone a telephone bill. But normalization of poll around the most plausible party affiliation should take care of that - the criticism that Gallup and some others are taking is that they’re not; they’re using unrealistic normalizations that favor the sponsor of the poll.

20

No, I don’t think it works this way. I think in general, pollsters call Registered Voters until they have x respondents, where x works out to be a little more than 1000 for a margin of error of around 3-4 percent.

In Gallup’s case, they also ask a series of questions meant to gauge the respondent’s likeliness to vote; these tend to be yes/no questions, and they add up the number of ‘yes’ answers for a given respondent to place them on a continuum of Likely Voters.

They then ask the meat of the poll: who are you voting for? What is your party affiliation? The results for all 1000 respondents are calculated and released as registered voter results.

Here’s where the controversy starts: These polls are allegedly of random registered voters. But if you release a poll that says that 40% of your respondents are Republicans, when (as I have said) a poll of 19,000 voters that ended in July of this year says that only 29% of them identified as Republican, then this poll becomes hugely suspect. It could, certainly, be that 40% of all Americans do identify themselves as Republican; but the pollster would have to explain how that figure could have jumped 11 percent in seven weeks, when it historically fluctuates no more than 6 points over a period of years.

Here’s the second part of the controversy: Registered Voters versus Likely Voters. LVs are a subset of RVs, and LV’s are measured based on their responses to the set of likeliness-to-vote questions. (青汁を飲む時間で効果は変わる？ has a list of the seven questions used, and a better explanation than I could come up with for how they’re used.) Basically, the higher your score is, the more your response is weighted. They toss out the lowest y number of responses based on projected turnout; recently they’ve been estimating a 55% turnout. I’m still trying to get a number on the 2000 turnout, I’ve heard it was as high as 66%, which makes Gallup look like a joke, to say nothing of the widely held belief that turnout this year will be even higher.

That same cite also compares the accuracy of Gallup’s likely voter model to recent presidential elections and finds it lacking; the registered voter numbers have been more accurate 3 out of the last 4 elections.

Bottom line is: believe Gallup if you want, but it is showing results found among no other polls and its methodology has been shown to be suspect.

This race is essentially tied, and the only votes that matter are the state polls for the Electoral College.