Party Identification discrepancy. Why?

[DSeid]

The issue has come in at least two threads: there is a significant discrepancy between party identification results in the Gallup and Rasmussen numbers (which show stable near parity voter ID in the case of Gallup or a sudden increase in GOP ID in the case of Rasmussen) and the consensus of the other polling houses as summarized graphically by HuffPo, which show Democratic ID stable and Independent increasing at the expense of GOP ID.

Anyone with a GQ level answer of why?

(Oh sure I know the speculations, from lack of cell phone sampling to incorrect assumptions used to “correct” the demographics of the samples, but I am wondering if anyone actually has actual facts to bring to bear.)

[TriPolar]

@DSeid

(Oh sure I know the speculations, from lack of cell phone sampling to incorrect assumptions used to “correct” the demographics of the samples, but I am wondering if anyone actually has actual facts to bring to bear.)

I don’t think the reliance on landlines for conducting polls is simply speculation. I don’t have the sources available, but several pollsters have admitted having a hard time finding cases to fill out their polls because people aren’t answering landlines.

[DSeid]

A Daily Kos article on the issue.

Rasmussen can probably just be dismissed as a biased and historically inaccurate house.

Gallup though …

Gallup is the oldest and one of the most respected polling organizations in the world. Unlike Rasmussen, they provide a great deal of information about their sampling methods and share their results and at least some of their data with interested scholars. Gallup also uses what is generally considered state-of-the-art polling methods. They conduct their interviews using live telephone callers and they supplement their landline sample with live calls to cell phones. Currently about 30 percent of Gallup’s interviews are conducted on cell phones. …

… In 10 polls this year, Gallup has found an average Democratic advantage in party identification among the voting age population of only 2.1 percentage points. Other polling organizations using similar methods have produced an average Democratic advantage in party identification of 7.8 percentage points since the beginning of 2012. That is based on 28 polls conducted by 7 polling organizations: CBS/New York Times, ABC/Washington Post, Ipsos/Reuters, NBC/Wall Street Journal, Pew, Kaiser and AP/GfK.

It is much harder to explain the Republican skew in Gallup’s polling on party identification than it is to explain it in Rasmussen’s polling. Gallup’s party identification question is different from the questions used by most other polling organizations in that it references “politics as of today” rather than focusing on a longer-term party preference. That may make it somewhat more responsive to current political issues. But that would not explain why Gallup also appears to have a similar Republican skew in its presidential preference polling. Another factor may be the racial composition of Gallup’s sample which may include a slightly smaller proportion of nonwhite respondents than other polls using similar methods although such comparisons are difficult to make because not all polling organizations provide information about the racial composition of their samples.

So not too much help but it rules out the cell phone explanation for Gallup.

Thing is that Gallup is a consistent measuring device … whatever error they have should be a consistent one. And they show no significant moves from GOP to independent while other also reliable firms do. Is there a demographic change that they are not accounting for or one that “the rest” are falsely factoring in?

[Merneith]

Nate Silver on cellphone polls:

http://fivethirtyeight.blogs.nytimes.com/2012/09/19/obamas-lead-looks-stronger-in-polls-that-include-cellphones/
He also has a related article showing how Gallup is probably an outlier at the moment.
http://fivethirtyeight.blogs.nytimes.com/2012/09/21/sept-20-obamas-convention-bounce-may-not-be-receding/#more-34814

The short answer seems to be: Gallup is a reputable polling firm, but it’s only *one *polling firm.

[DSeid]

Gallup uses cell phones (see above).

Not asking about the at the moment possible “statistical variance” of the snapshot polls, but the long term consistent trend that Gallup has of giving distinctly different party ID numbers than do the rest.

They are just one but a respected one with a long track record and a consistent tool. They are doing something consistently differently then the rest. Sure they can be wrong, perhaps likely are, but one, they might be the one who has it right, and two, I’d still like to know what that something different is, before judging if the mistake is theirs or one that everyone else is doing systematically, and just because the puzzle interests me. It may inform some about the process and the electorate.

[Untoward_Parable]

People are embarrassed to call themselves Republican. They may vote that way anyway out of shortsighted greed while being disgusted by the social politics or vice versa but not want anyone to think they agree with the full platform.

[Untoward_Parable]

The Democratic party is distressing because they often don’t follow their advertised ethics and philosophy but the Republicans advertised ethics and philosophy are disgusting even when they’re being honest.

[DSeid]

For those who are interested I found this detailed HufPo article attempting to understand the Gallup house effect. (Not exclusively for party ID.)

For estimates of race and other demographics, pollsters typically turn to the highly reliable data produced by the U.S. Census. They sometimes look at the full Census, but since that occurs only once every 10 years, many depend instead on the more frequently fielded Current Population Survey or the American Community Survey. …

… Gallup currently weights its adult samples to match the March 2011 CPS, the largest monthly CPS sample that is currently available to the public.

Gallup adds an unusual twist, however, which most other widely reported independent national polls do not: It filters the CPS data to use the estimates for adults “living in U.S. telephone households.” Thus, rather than weighting its survey to match the demographics of all adults, Gallup removes a small sliver of adults (3.2 percent) who have no access to telephone service of any kind, either landline or mobile.

The omission is consequential because the majority of Americans who lack all forms of phone service are non-white. In the March 2011 CPS, 54.4 percent of adults living in non-telephone households reported they were Hispanic, black or another race other than white, compared to 31.5 percent of adults with telephones. …

… Most other pollsters have opted to go in a different direction, basing their weighting targets on the full population, since that is ultimately what their polls of “all adults” endeavor to measure. By filtering out the non-telephone households, Gallup slightly underestimates the number of blacks and Hispanics in the full population. …

… The way Gallup asks its survey respondents about their race is also unusual. … In effect, Gallup is not weighting its black respondents as heavily as it should because it has used a slightly broader filter that gives it a higher percentage of “black” respondents to start with. …

… Pollsters like Gallup attempt to balance a number of variables at the same time – race and Hispanic ancestry, gender, age, education, region, number of adults in the household and telephone usage (mobile or landline).

A common method for weighting by multiple variables, called “raking,” involves weighting the data by one variable at a time, in sequence and repeating as necessary, until the weighted percentages for the various demographics match their targets (or come reasonably close) … As it turns out, Gallup’s weighting often falls short when it comes to hitting the targets for race and Hispanic ancestry. …

… Considered alone, any one of the three issues raised here – the weighting targets set by Gallup, the different question format used to ask about race and the failure of the weight targets to hit their mark – would probably not make enough difference to affect the results for questions about presidential job approval or vote preference. But a focus on all three issues combined does help explain differences in the results among various pollsters. …

…** If we recalculate the Obama job rating for the seven USA Today/Gallup surveys based on a racial composition that matches the Pew Research surveys conducted over the same period, the evident house effect differentiating the two pollsters on Obama job approval is largely erased**. …

Bolding mine.

So Gallup uses a method that results in a different racial mix than others do, which by this analysis underweights minorities, especially urban minorities. That difference may be enough to explain their house effect on various items. FWIW.

[DSeid]

Last night’s 538 promises to adress the Gallup house effect soon: “… his numbers held steady in the Gallup tracking poll, which continues to show a tied race. (We’ll have more about how to think about that Gallup poll in a separate article.)”

It should be edifying.

[TriPolar]

I don’t have cites, just a gathering of tidbits from political reporting, so take it all with a grain of salt:

1. The party association shift. More of those now calling themselves Independent used to identify as Republican, and more of those now calling themselves Democrats used to identify as Independent.

2. Cell phones. I don’t know if this explains Gallup, but the cell phone pollees aren’t well calibrated yet. There’s a lot of adjustment to the numbers, and cell phones are an area where they don’t have a lot of successive polls to measure the accuracy of the adjustments. This happens whether cell phone users are polled or not.

3. General decreased poll response rate. All of their calculations may be off by a lack of responses across many categories.

4. Political propoganda. This could be nothing but a common deviation that has been highlighted by political reporting and isn’t a real concern.