# New Pew Political Typology Poll: Terribly Designed, or Just Me?
**URL:** https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793
**Category:** Great Debates
**Created:** [June 27, 2014, 5:06pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793 "2014-06-27T17:06:06Z")
**Posts on this page:** 17
**Page:** 1
### Author:  [@Richard\_Parker](https://boards.straightdope.com/u/Richard_Parker)
#### Post date: [June 27, 2014, 5:06pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/1 "2014-06-27T17:06:06Z")
The new [Pew Political Typology poll](http://www.people-press.org/2014/06/26/the-political-typology-beyond-red-vs-blue/) has the political blogs putting out the usual punditry about the state of American politics. But if you look at the poll itself, the design is awful. It’s constructed as a series of “which comes closer” questions–which is fine–but a majority of the questions aren’t even on two ends of the same continuum. For example:
> [@](#):
>
> Government regulation of business is necessary to protect the public interest
>
> Government regulation of business usually does more harm than good
This is a statement of principle paired with an empirical statement, and not only are they different in kind, they aren’t even opposites. It is perfectly possible (and in my view, correct and accurate) to agree equally with both statements. Or consider this abomination of logic:
> [@](#):
>
> Most people who want to get ahead can make it if they’re willing to work hard
>
> Hard work and determination are no guarantee of success for most people
It is possible (and indeed, true, in my opinion) for most hard workers to be able to make it, but for hard work and determination to be no guarantee for most people. They are not mutually exclusive statements, and neither is “closer” to the truth.
The best you can say for this kind of survey design is that people who tend to be X tend to pick Y answer. And that’s fine if your goal is just to classify the poll-taker into a pre-defined category. But that’s not what Pew purports to do! They also purport to be determining what these categories believe at the same time. For example, consider this binary:
> [@](#):
>
> Poor people today have it easy because they can get government benefits without doing anything in return
>
> Poor people have hard lives because government benefits don’t go far enough to help them live decently
Now, I don’t think either of these is true. If forced to choose, I might pick 2 just because I know that’s the “liberal” answer, and because I resent the “without doing anything in return” part of the first question. But if you then concluded that liberals believe “Poor people have hard lives because government benefits don’t go far enough” you’d be wildly mischaracterizing my view and that of many liberals (and the same is true for those picking the conservative answer). But that’s exactly what Pew, and the many reporters using this poll for their stories, have done.
\_\_
So what am I missing here?
---
### Author:  [@puddleglum](https://boards.straightdope.com/u/puddleglum)
#### Post date: [June 27, 2014, 7:45pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/2 "2014-06-27T19:45:51Z")
In designing a survey the trick is to make both options equally appealing. That way you can get a approximation of what the person thinks and not what the person thinks you want them to say. The survey seems to have done a pretty good job doing this. Putting people into distinct groups only makes sense if there are discreet differences between the groups, I don’t see that there is, but I could be missing something.
What you can’t use this for is like an opinion poll. If 50% of people pick option A then you can’t say option A is actually what 50% of the people think. Lots of people are ignorant about survey design and usage so you can bet lots of ignorant things will be said about the results.
---
### Author:  [@Happy\_Fun\_Ball](https://boards.straightdope.com/u/Happy_Fun_Ball)
#### Post date: [June 28, 2014, 1:29pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/3 "2014-06-28T13:29:39Z")
It looks to me like they have tried to correlate the answer on certain questions with results of their larger political typology poll. Kind of like the way the statisticians at OKCupid found that compatibility in a relationship can be boiled down to how you answer 3 questions:
1. Do you like horror movies?
2. Have you ever traveled around another country alone?
3. Would it be fun to give up everything and go live on a sailboat?
It does not matter, statistically at least, how you answer any other question. Even questions about religion or morality don’t really correlate with compatibility like these three questions do.
Presumably the folks at Pew have done something similar. Whether your answers really tell someone anything about your Typology may be beside the point, if you answer these questions a certain way it statistically places you within a group.
Who knows though? I could be wrong. I do know that the last time they did this it was awful. I went through the quiz several times and it was seemingly impossible, no matter how you answered, to place yourself in a particular group. I did not really agree with the group i was placed in when running though the types and thought my opinions were more aligned with other groups. This year I agree with where they put me.
---
### Author:  [@BrainGlutton](https://boards.straightdope.com/u/BrainGlutton)
#### Post date: [June 28, 2014, 2:45pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/4 "2014-06-28T14:45:59Z")
Bear in mind, Pew has been doing this every few years since 1987 (and every time they do it, they come up with a new set of typologies). I’m sure they’ve learned a few things about survey design over the years.
---
### Author:  [@BrainGlutton](https://boards.straightdope.com/u/BrainGlutton)
#### Post date: [June 28, 2014, 2:51pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/5 "2014-06-28T14:51:17Z")
[This is interesting:](http://www.people-press.org/2014/06/26/the-political-typology-beyond-red-vs-blue/)
> [@](#):
>
> **The More Things Change…**
>
> The Pew Research Center created its first political typology in 1987, when President Ronald Reagan was reeling from the Iran-Contra affair and the front-runners for the upcoming presidential election were Sen. Gary Hart for the Democrats and Vice President George H.W. Bush for the GOP.
>
> Much has changed in politics over the past 27 years, of course. But some of the same fissures we identified in that first typology are still evident today. Today, **Business Conservatives** hold much more liberal positions on homosexuality and morality than do the other largely Republican group, **Steadfast Conservatives.** In 1987, **Enterprise Republicans** were much more tolerant than **Moral Republicans** on many of the social policy issues of the time, such as whether or not school boards should have the right to fire homosexual teachers.
>
> On the left today, the majority-non-white **Faith and Family Left** is highly religious and has much more conservative attitudes about the acceptability of homosexuality and abortion than do the **Next Generation Left** or **Solid Liberals.** But this same fissure existed in the late 1980s between what we then described as the **Partisan Poor** and **Passive Poor** and the more upscale **60s Democrats** and **Seculars.**
>
> Every Typology study between 1987 and 2014 has found a very downbeat, financially stressed group who is skeptical of government, yet supports the programs that assist people facing economic disadvantages. Over time, we have referred to them as the **Disaffected,** the **Embittered** or today the **Hard-Pressed Skeptics,** and their political leanings have varied with administrations, issues and economic conditions.
>
> But not all has been constant in the Political Typology, reflecting changing generational, demographic and political contexts. An anchor of the Democratic Party in 1987 was the **New Dealers,** representing the generation raised during economic strife and World War II. With the New Deal generation largely gone, the Democratic coalition now leans heavily on the **Next Generation Left,** who are more socially liberal than their predecessors, but also somewhat more economically conservative.
>
> And the appearance of the **Young Outsiders** is a new group in the political landscape. Previous typology studies have found right-leaning groups, such as the **Pro Government Conservatives** in 2005, who proved critical to George W. Bush’s reelection. Drawn to the GOP coalition by the party’s positions on security issues and morality, they otherwise supported a more expansive view of government and social programs. The **Young Outsiders** today are very different, as they share the GOP base’s deep skepticism of government programs, but favor a more limited foreign policy, and hold decidedly liberal social views.
---
### Author:  [@Chronos](https://boards.straightdope.com/u/Chronos)
#### Post date: [June 28, 2014, 3:13pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/6 "2014-06-28T15:13:54Z")
How is it useful to change categories every year? You’d think it’d be more valuable to keep mostly the same categories year to year, so you can track what’s happening across time. **BrainGlutton** ’s quote seems to be trying to do this, but the “business conservatives”, say, aren’t the same thing as the “enterprise Republicans”.
---
### Author:  [@Flyer](https://boards.straightdope.com/u/Flyer)
#### Post date: [June 28, 2014, 4:30pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/7 "2014-06-28T16:30:31Z")
> [@Richard\_Parker](#):
>
> [snip]
> So what am I missing here?
What you’re missing is that you don’t have a clue how surveys are designed.
First–the first pair you mentioned, any traditional conservative and every libertarian will reliably pick the second one, and any traditional liberal will reliably pick the first one.
The second pair is slightly iffier, but most conservatives and libertarians will pick the first statement, with most liberals picking the second statement.
The third pair is also pretty clear-cut; most conservatives and libertarians pick the first, and all liberals pick the second.
Those three pairs of statements are sufficient in and of themselves to accurately categorize the broad political views of most politically active adults. Anything else is simply refining the picture.
Edit: I conducted telephone surveys for over a year. The listed examples are perfectly normal, and widely used by multiple companies. As a matter of fact, I probably did at least one survey that had those three pairs word-for-word.
---
### Author:  [@Captain\_Amazing](https://boards.straightdope.com/u/Captain_Amazing)
#### Post date: [June 28, 2014, 4:43pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/8 "2014-06-28T16:43:49Z")
Aren’t these two mutually exclusive? I don’t know what your complaint is.
> [@](#):
>
> Most people who want to get ahead can make it if they’re willing to work hard
>
> Hard work and determination are no guarantee of success for most people
1 says that, for most people, if they work hard, it’s a guarantee they’ll succeed.
2. says that for most people, if they work hard, there’s no guarantee they’ll succeed.
---
### Author:  [@Richard\_Parker](https://boards.straightdope.com/u/Richard_Parker)
#### Post date: [June 28, 2014, 5:33pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/9 "2014-06-28T17:33:29Z")
> [@Captain\_Amazing](#):
>
> Aren’t these two mutually exclusive?
No, they aren’t. It is entirely possible that most people who work hard get ahead without hard work being a guarantee for anyone.
---
### Author:  [@Richard\_Parker](https://boards.straightdope.com/u/Richard_Parker)
#### Post date: [June 28, 2014, 5:34pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/10 "2014-06-28T17:34:38Z")
> [@Flyer](#):
>
> Those three pairs of statements are sufficient in and of themselves to accurately categorize the broad political views of most politically active adults. Anything else is simply refining the picture.
I think you’re misreading my complaint. As you’ll see, I acknowledged exactly that point in my original post. To quote myself:
> [@](#):
>
> The best you can say for this kind of survey design is that people who tend to be X tend to pick Y answer. And that’s fine if your goal is just to classify the poll-taker into a pre-defined category. But that’s not what Pew purports to do! They also purport to be determining what these categories believe at the same time.
---
### Author:  [@BrainGlutton](https://boards.straightdope.com/u/BrainGlutton)
#### Post date: [June 28, 2014, 11:13pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/11 "2014-06-28T23:13:23Z")
[Here’s](http://www.people-press.org/2014/06/26/appendix-2-about-the-political-typology/) what the Pew Research Center says about the poll-design:
> [@](#):
>
> The 2014 political typology divides the public into seven politically engaged groups, along with an eighth group of less engaged Bystanders. The assignment of individuals to one of the seven core typology groups is based on their responses to 23 questions about social and political values.
>
> The questions used in the typology construction are in a balanced alternative format where respondents choose which of two statements most closely reflects their own views (Q25 and Q50 in the topline). Items included in the typology construction measure a variety of dimensions of political values and are used to group people in multi dimensional political space. Many of these values are similar to those used in past typology studies; however this year’s typology is a departure from past analyses because it uses individual questions; in prior years, scales measuring underlying political dimensions were constructed using many of the same questions. This new approach allowed for the inclusion of some additional values not used in prior typologies (for instance, values about U.S. involvement in world affairs). In addition, unlike previous typologies, this year’s typology does not include any measure of party affiliation in the model, nor does it include measures of personal financial security.
>
> The typology groups are created using cluster analysis, a statistical procedure that uses respondents’ scores on all 23 items to sort them into relatively homogeneous groups. The tables on the following pages show each of the questions and the distribution of responses among of each of the seven typology groups (excluding Bystanders).
>
> Cluster analysis is not an exact process. Different cluster solutions are possible using the same data depending on model specifications and even the order in which respondents are sorted. In order to address the sensitivity of cluster analysis to the order in which cases are entered, each cluster model was run several thousand times, and the results compared, to identify the solution that produced the set of groups that were both homogeneous internally and different from one another with respect to the set of political values. In technical terms, the solution for each model with the lowest sum of squared error of the clusters was chosen. Models with different numbers of clusters were examined, and the results evaluated for their effectiveness in producing cohesive groups that were sufficiently distinct from one another, large enough in size to be analytically practical and substantively meaningful.
>
> While each model differed somewhat from the others, all of them shared certain key features. The final model selected to produce the political typology was judged to be strongest from a statistical point of view, most persuasive from a substantive point of view, and was representative of the general patterns seen across the various cluster solutions.
>
> As in past typologies, a measure of political attentiveness and voting participation was used to extract the “Bystander” group, people who are largely not engaged or involved in politics, before the remaining respondents were sorted into groups. Bystanders are defined as those who are: 1) Not registered to vote; 2) Say they seldom or never vote; and 3) Do not follow government and public affairs most of the time. They represent 10% of the overall population and were held aside prior to scale development and assignment of the remaining 90% of respondents to their typology groups.
---
### Author:  [@rsa](https://boards.straightdope.com/u/rsa)
#### Post date: [June 29, 2014, 1:13am UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/12 "2014-06-29T01:13:13Z")
Next Generation Left (even though I’m mid-fifties). I didn’t answer questions 4, 12, and 14. I wonder if no answer was allowed in the original survey. If not answering is intended to be allowed, then they should state that. Otherwise it is unfair, in that only intelligent people (mostly liberals) would consider the possibility of no response. 🙂
ETA: Yes, I think this is a horrible design. I’m sure there’s some liberal, newfangled justification to it, but it just a’int right darn it.
---
### Author:  [@Hector\_St\_Clare](https://boards.straightdope.com/u/Hector_St_Clare)
#### Post date: [June 29, 2014, 5:30pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/13 "2014-06-29T17:30:14Z")
It’s terribly designed.
I answered down-the-line left on the economics issues, environmentalist on the environmental issues, and isolationist on foreign policy, but the social questions were tough. What happens if you favour immigration from some parts of the world but not others? What do you say if you don’t think the problems of African Americans are their own fault OR due to racism? How do you respond if you don’t especially care about the marriage rate, but care a lot about the fertility rate?
Hard Pressed Skeptic, for what it’s worth.
---
### Author:  [@Pleonast](https://boards.straightdope.com/u/Pleonast)
#### Post date: [June 30, 2014, 7:28pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/14 "2014-06-30T19:28:51Z")
We complain about this every year.
Their basic point
> [@](#):
>
> Even in an increasingly Red vs. Blue nation, the public’s political attitudes and values come in many shades and hues.
is correct. Their problem is that that’s true, which makes it very hard to classify people.
They’ve chosen their categories and they match people to them. They classified me as “Next Generation Left”. When I look at their “Ideological Placement”, I came in above “Mixed” with the bulk of my category to my left. So not a good fit. But it’s nice to see the tails of their categories; it’s a good indicator of the fuzziness of their groups. What I’d really like to see are numbers about how well I fit into each group, instead of a single “this is your best fit”.
The trouble, as has been pointed out, is that the questions have binary choices. There’s no weighting based on priority or intensity. They have no way to distinguish the cases of 1) I strongly agree with statement A and disagree with statement B vs 2) strongly agree with A and somewhat agree with B vs 3) disagree with A and strongly disagree with B. I pick A in each of those cases, but my beliefs are very different.
---
### Author:  [@Richard\_Parker](https://boards.straightdope.com/u/Richard_Parker)
#### Post date: [June 30, 2014, 7:32pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/15 "2014-06-30T19:32:07Z")
I’m not even really complaining about the use of the poll to place people into pre-defined buckets. That has some issues but seems more or less methodologically sound.
My chief complaint is that they then purport to say what those buckets believe, based on the answers to the binary questions. I see no basis at all for that conclusion. All you can reliably conclude is that X group thought Y answer “came closest” to their belief. It is just a logical travesty to conclude that X group believes Y (or that Z% of X group believes Y).
That makes nonsense out of all the reporting about how conservatives think the poor have it easy. The people reading the poll that way, including Pew, don’t have a leg to stand on.
---
### Author:  [@astorian](https://boards.straightdope.com/u/astorian)
#### Post date: [June 30, 2014, 8:47pm UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/16 "2014-06-30T20:47:12Z")
I agree with Mr. Parker. Someone referred me to the quiz, ands I found that, as often as not, I agreed with both statements or disagreed with both. I gave up the quiz long before I got to the finish, because I couldn’t see the point in continuing.
James Madison said it better than I could: _ **"If men were angels, no government would be necessary. If angels were to govern men, neither external nor internal controls on government would be necessary. In framing a government which is to be administered by men over men, the great difficulty lies in this: you must first enable the government to control the governed; and in the next place oblige it to control itself. A dependence on the people is, no doubt, the primary control on the government; but experience has taught mankind the necessity of auxiliary precautions."** _
Is that a liberal or conservative philosophy?
Would the man who wrote that be able to choose blithely between “People should be free to do what they want without government interference” and “People need strong government”?
Neither can I.
---
### Author:  [@BrainGlutton](https://boards.straightdope.com/u/BrainGlutton)
#### Post date: [July 1, 2014, 12:05am UTC](https://boards.straightdope.com/t/new-pew-political-typology-poll-terribly-designed-or-just-me/691793/17 "2014-07-01T00:05:08Z")
> [@astorian](#):
>
> James Madison said it better than I could: _ **"If men were angels, no government would be necessary.** _
He was wrong about that, of course. Many things make government necessary that have nothing to do with the human tendency to misbehave. E.g., we need it to provide many public services other than policing.