Goddamn, that should not have happened. Meant to preview.
Continuing…
So the problem with this study is that it merely evaluates a correlation between the erstwhile failure of the American public to absorb information and a particular electoral behavior. Fine, this is pretty straightforward. An eighth grader can calculate the correlation between a vote for Bush and ignorance of the Duelfer report. The study’s fifteen some-odd pages of findings are very tedious, and the graphics aren’t even very good.
The real problem with this study is not in its argument but in its implicit suggestion that you may use this sample to somehow draw inferences about the population. The researchers frame this study in a series of questions precipitated by “How do Americans feel about…”, and suggest that they can answer these questions with their study. They try to infer the effects of ignorance on electoral behavior.
This is incorrect and misleading. The researchers are essentially arguing by apposition: if a bunch of Bush voters were ignorant, then the rest are, too.
It is very possible to make this claim and it would not be difficult to design a study to test it. The quantity of interest is the voter’s electoral behavior. It is possible to present a mathematical model which argues that ignorance, which can be quantified, affects a voter’s presidential choice. One can test this model with a sample, and report the magnitude of the effects (the coefficients of the independent variables), the standard errors, and the statistical significance. However, any time one chooses a mathematical model, the modeler has to make certain assumptions about the real world. Sometimes these are reasonable, sometimes less so. Often if the assumptions are relaxed, the model falls apart entirely. Often models can be criticized on the grounds of endogeneity or omitted variable bias. These are very serious and can undermine the inferential value of a model.
The authors are trying to answer a question about the American population without actually presenting a model for scrutiny. This abrogates them of the responsibility of presenting coefficients, standard errors, and confidence intervals. It also prevents the assumptions of their model, which usually have serious real world implications, from being challenged. Their correlations offer us no way whatsoever to predict how a voter of certain characteristics will behave. The only statistic they report, the so-called “margin of error”, is thoroughly uninformative. Finally, to my irritation, they do not even include the data set. I would have loved the opportunity to run some of the numbers myself.
The study is not unhelpful because it is somehow politically biased or unscientific. Knowledge Networks is an excellent outfit, and I believe they conduct these surveys with considerable professional integrity. The study is garbage because all it tells us is how 968 respondents answered a survey. This is in no way informative about the state of the American electorate.
I would be happy to go into more technical detail if anyone wants. I do not want anyone to think that I am just handwaving. If you are satisfied with my analysis, then there is no point in making myself out to be one of those condescending, pseudo-intellectual northeasterners. Alternatively, you can check out this Wikipedia article that should give you a nice summary of statistical inference and theory.
If you are interested in more technical pieces on binary dependent variable models, the literature is huge. I would be happy to suggest some titles.