The PIPA Report: Americans on Iraq on the Eve of the Presidential Election

Great! Thanks!

-XT

Sure, just like if you drive back a black sheep, all you can say is that it is black on one side since you didn’t see both sides of it. Technically, the correlation seems to be between candidates’ supporters and the knowledge or ignorance on certain issues. It is true that it doesn’t say what is true about the people who actually vote. However, one might imagine that since a reasonable percentage of eligible voters do vote, there won’t be huge differences amongst the population that actually does vote. It would be quite perverse to posit a situation in which the large correlation seen is substantially altered when you consider only actual voters. [And, indeed, while pollsters like Gallup who distinguished between registered voters and those likely to vote do their correction, it shifts the results in polls of Kerry vs. Bush somewhat but not a huge amount.]

I agree that we need to be careful about confusing correlation and cause. The survey in question shows only that there is a significant positive correlation among potential voters between ignorance on certain issues and support for Bush. We can’t draw the conclusion that the ignorance causes the support for Bush.

However, I disagree with the argument that a study that shows only correlation and not cause is useless or counterproductive. Heck, there are major medical studies that show only correlation without identifying any causal mechanism, but the results are still quite significant.

Studies of causality are good too, though. If anybody wants to draft a sample survey that purports to test whether there really is a causal link between ignorance on particular political issues and support for a particular candidate, I’d like to see what they come up with.

Maeglin:

I don’t think that correlation studies have to be intentionally misleading when it comes to politics. As long as no one is trying to pronounce as fact that ignorance is the cause of Bush-support (and vice versa), I don’t see what the problem is.

The interesting thing about this study is that it demonstrates that Bush and Kerry supporters aren’t simply the same people except with different political preferences; they are significantly different with respect to awareness. However, it is very true that there may be major confounders at work. Kerry-supporters may have more time on their hands to read newspapers and watch TV. Bushites may be older than Kerryites and thus, less likely to engage in informative internet discussions such as this one (just speculation). Both leisure time and age may be factors more closely associated with voter preference than awareness, and the seeming associating between awareness and voter preference is just the by-products of the other factors. However, that doesn’t take away from the studies’ conclusion or imply that the results are “intentionally misleading”. It just means there may be other factors at work that need to be controlled for.

This is a good point, but keep in mind that this is not a study attempting to prove that ignorance leads to Bush-support. Temporality of the outcome and factor can not be establishment by polling. In a cohort study (which flows forward in time) designed to test whether a drug produces a certain side-effect, you most certainly can show that drug X leads to outcome Y, by contrasting the test group with a placebo group. The PIPA study, by design, was never intended to do that. Don’t want you to compare apples and oranges.

Kimstu

I think in order to establish cause, the focus would have to be on the study itself, not the survey.

There’s two ways we could go about it. The first way, which is cheaper but establishes causality less confidently, is a case-control study. Find 1000 people who voted for Bush (case group) and then compare their ignorance levels with, say, 4000 people who did not vote for Bush (control group). This would be done via an odds ratio. If ignorance leads to Bush-support you would expect to see an odds ratio greater than 1, suggesting that Bushites are X as likely to be ignorant than Kerryites.

Of course, in this situation, pains would have to be taken to control for confounding variables (age, socioeconomics, race, etc) before concluding that ignorance leads to a certain outcome.

The second way we could go about it is by a cohort study. For our study sample we’d use 1000 people. One caveat: they’d have to be completely and utterly nonpartisan (good luck with that one!). The sample will then be split into two groups: the “informed” and the “uninformed”. Both groups will have identical characteristics except half the subjects will be unindated with infomation from diverse media outlets via newspapers, magazines, TV, radio, internet, etc. The other half will have only restricted access to information: no newspapers, no internet, little TV. After four years of the experiment, we’d quiz each subject on current events (Iraq, 9/11, Hussein) and we’d also ask each subject who they want to vote for.

If more Bush-supporters come out of the “uninformed” group (secondarily confirmed by the quiz), then it suggests a causal relationship exists between ignorance and voter preference.

This is what I think upon encountering this survey:

  1. I think we all think that a person’s vote should be based on true beliefs about the candidates.
  2. Corollary to this, I think we all think that a vote for a candidate not based on true beliefs about that candidate is somehow defective.
  3. Pipa’s study shows that the majority of Bush’s supporters have false beliefs about some key facts regarding Bush as a candidate.
  4. From 2 and 3, I can conclude that a majority of Bush votes were somehow defective.

Taking this conclusion as established, I go on to think “Were I a Bush voter, I would have a prima facie case that I ought to look into the possibility that my own vote was defective in that it was based on false beliefs about Bush as a candidate.”

These conclusions, and I think nothing else significant, are what I take away from the survey. Should I not take even these away from the survey? Are there other things I can take away from it?

Thanks for your help.

-FrL-

You have done what Magelin said the paper was designed to get you to do. We have no evidence that the people actually voted based on their understanding of these issues. Correlation is not causation. Being massively uninformed about a candidates stand on issue X is no big deal if you are a single-issue voter and that issue is issue Y, or Z, or Q, or F. These votes can in no way be considered “defective” because the issue they were misinformed about was not part of the reason they supported Bush.

The study seems designed to heavily imply that Bush supporters supported him BECAUSE they were ignorant of the actual situations, but the data needed to draw that conclusion using the tools of statistical sampling was not gathered or analyzed in a way which would settle that question.

I would not recommend taking those conclusions. They may be true, but this data doesn’t prove it. What you can take away is that Bush supporters tend to be less informed or outright misinformed on these issues compared to Kerry supporters. The only action plan I would recommend is a massive campaign of information spreading about the facts in Iraq and about the policies/positions of Bush in general.

Enjoy,
Steven

Let me restate my argument:

  1. I think we all think it is safe to assume that people’s votes are in fact based on their beliefs about the candidates’ positions. (So let’s take it as given.)
  2. I think we all think that a person’s vote should be based on true beliefs about the candidates’ positions. (So let’s take it as given.)
  3. Corollary to this, I think we all think that a vote for a candidate not based on true beliefs about that candidate’s position is somehow defective.
  4. The questions listed on Pipa’s survey were sufficiently representative of possible beliefs about candidates’ positions.
  5. Pipa’s study shows that the majority of Bush’s supporters have false beliefs about some key facts regarding Bush as a candidate.
  6. From 1, 3, 4 and 5, I can conclude that a majority of Bush votes were somehow defective.

I think this argument is valid, and I think it’s what I really meant by what I said before. Notice I don’t say the study establishes that beliefs strongly determine votes, I just take that as an outside assumption–but a very safe one. Indeed, it seems to me it would be odd to think otherwise.

Now, I say the argument is valid, but perhaps it’s not sound. One of the assumptions might be wrong–I think the best candidate is probably number 4. Maybe that’s what you guys have been getting at all along?

But don’t misunderstand me to be saying that the study establishes a causal phenomenon in and of itself–clearly the study shows a correlation, but by itself, no causal relationship. I wasn’t saying the study shows a causal relationship.

The conclusion of the argument is that a Bush vote is probably defective, where 'defective" means “based on incorrect beliefs.” The study’s role in my argument is not to show that votes were based on incorrect beliefs–it is only to show that there is a correlation between incorrect beliefs and Bush votes. The “based on” part is brought into the argument explicitly as an assumption not based on the study.

-FrL-

One other point–from the study by itself, I still think we can conclude that:

“If I am a Bush voter, then there is a very good chance I am very misinformed, at least on certain issues.”

And if I am indeed a Bush voter, then this should be a significant fact. And I think you just need the study to learn that fact, none of the outside assumptions (however plausible) I mention in my previous post.

-FrL-

Actually, if their hypotheses in the analysis section of the report are more complicated than that and if anything go the other way:

I do agree that these explanations are in the realm of conjecture. But I also don’t think that “Bush supporters supported him BECAUSE they were ignorant of the actual situations” is a very good summary of the above arguments.

Well, if I read it right, they are more suggesting in spite of rather than because of. And I think their assessment that the tenacity of their affection does indeed result from a “bonding” experience born of trauma makes all the sense in the world. This at least adds a human dimension to what is otherwise perplexing. If I may indulge an understatement, perplexing.

I didn’t see what they saw. He went where he was supposed to go, said the things he was supposed to say. The role of the leader in this situation is scripted, we knew what we expected, he knew what we expected. It could have been filled from Central Casting. He took no risks, committed no heroisms. All he needed to do was follow the mutually accepted script, and he would be swamped in approval and respect, nay, adulation.

We forget that just prior to that awful day, his approval was, at best, mediocre. He looked destined to muddle through a term without mucking it up too badly. Perhaps this is when he was siezed by the notion of himself as a Leader of Men, a collossus leaving his mark on history. Lincoln, Roosevelt, Bush…

So what I think is: they are as cognizant of the facts as any of us, they simply make an assumption we don’t make: that there is a hidden factor that would make it all turn right. The WMD were spirited away, Jane Fonda’s got 'em, something. Because they know that The Leader would not take them astray, therefore there must be some unknown factor.

It reminds of Reagan speech about Irangate: how he knew in his heart he was innocent, but the facts spoke differently. In essence, he apologized on behalf of the facts, how they had failed to reflect the deeper truth.

Well we all seem to agree that the interpretation of the results is somewhat subjective. But the results themselves remain solidly in their bunk despite the attempts to dislodge them therefrom.

Bush supporters were undeniably vastly more deluded about these factually incorrect statements, and this study suffers no more “problems” than substantially any political poll.

I find the results themselves shocking. To me, it appears that some Bush supporters are seeking a way to avoid accommodating them intellectually - something of an irony in itself since the study was largely all about cognitive dissonance.

This is the real issue. Some posters are reifying the results, as if somehow they express some independent truth or value free of any interpretation.

I disagree. A statistic is only as good as it is useful.

If these results fuel your indignation, hey, it’s your mental life. If you believe that a statistical result is informative and shocking if it cannot actually be used to infer testable, quantitative conclusions about the state of the world, then I’d say that perhaps you’re too easily shocked.

Further research is absolutely necessary before conclusions can be drawn. The authors of the study clearly do not feel that way, for their “analysis” consists of untested, scattershot speculations.

Like I said, if this is good enough for you, I rest my case.

Well, I did say that the interpretation was subjective. So, let us speculate some other explanations for those results in order that we may each choose the explanation which seems most reasonable to us.

I have yet to see any Bush supporters, or indeed anyone, forwarding anything like an alternative explanation. Perhaps they could, at the very least, point me to some equally clear examples of factually incorrect statements believed by Kerry supporters?

I’ll gladly admit my eyes glazed during much of that “analysis” section. I put the word analysis in quotations because I didn’t see much that I consider analytical rather than conjectural in that section. I’m open to criticism that I may have misinterpreted it, and I’m not going to defend my interpretation to the death or anything.

Still I don’t think my characterization was wholly wrong. I’ve snipped a large deal of the “analysis” quoted, everything but the piece which I think is the most critical. The piece which lead me to say they were implying a substantial fraction of electoral support(enough to “[succeed] in getting elected”) would be “based on false beliefs.”

Enjoy,
Steven

It all depends on the “truth” you’re trying to glean from the results, doesn’t it? If you’re trying to determine whether Bush and Kerry supporters are equally informed, the results of the study would tell you the answer to that question. If you’re trying to determine whether ignorance causes people to favor Bush over Kerry, then these results will not give you that answer.

The only “right” interpretation of the results is that which is answered by the analytical tool designed to answer the “right” question. Any other ideas derived from the results can only be speculation and should be presented as such. A Chi-square test, for example, will not tell you anything about causality, so infering it out of a correlation study is wrong. However, if the study had a cohort design and made use of the appropriate statistics (relative risk, etc.), it would be reasonable to infer the presence or absence of causality between two factors.

Why is it not useful to know there is significant difference in awareness between Bush and Kerry supporters? This is where you are losing people.

Can you tell me why the conclusion in this case is not testable or “quantitative”?

Conclusions can be drawn by this data; they just may be limited conclusions. If scientitists had to wait until every single possible hypothesis surrounding a problem were answered before making one conclusion, they’d never leave grad school.

As long as the speculations are presented as speculations, what is the problem? Without speculation, how would future hypotheses be generated? The researchers are only advancing possible explanations for the discrepancies found. This is what science is all about.

“Why is it not useful to know there is significant difference in awareness between Bush and Kerry supporters? This is where you are losing people.”

I second this. I think this is what I’m not getting.

-FrL-

If I may, I have been staying out of this discussion because, frankly, the statistics are beyond me. But I believe I can answer this question.

Because the question is not finished. If you want to ask “Do Bush supporters have less or more awareness about the Duefler report?”, then this poll seems to help. Are there any other more broadly based questions, however, that this poll can answer? Does it really point to a more general unawareness amongst Bush supporters, or does is simply point to an unawareness about a rather small handful of issues.

Does that answer the question?

** pervert:**

Since the questions in the poll more or less dealt with the contents of the Dueffler report, the study seems to answer the question “Do Bush and Kerry supporters have an equal understanding of the following issues?” pretty well, I think.

Or, more specifically, “Are self-identified Bush and Kerry supporters equally likely to be wrong about their candidates positions on this set of issues?” A question the chi-square analysis of the data would answer well.