I’m sure this could well end up in GD, so let me try to be very precise in what I’m asking.
As we all know, recently the medical journal the Lancet published this article. (Free registration req’d to read full text) The article used household surveys to conclude that 100,000 Iraqis have died as a result of the war, due to a great number of causes: from airstrikes to increased crime rates, heart attacks to infant deaths, and so on. At its core, this study appears to have more in common with a public health investigation than a question of body counts and arithmetic.
However, I do not understand the method by which the report was carried out. From those who understand statistics, surveys, public health, and polling better than I do:
Is this method of carrying out the study considered methodologically sound?
Is there any reason to suspect that this method of surveys results in distorted data, especially when conducted in a politically charged atmosphere of a war zone?
Have other notable public health studies used similar methods to try to establish mortality rates?
Have public health experts, mathematicians, or others weighed in, either for or against, the validity of the methods used in this study?
I realize there is a likelihood that some might expound, one way or another, about the inflammatory headline that 100,000 Iraqis have been killed. I am not interested in hearing about the results of the study, I’m only interested in the validity of how the study was carried out.
Based upon my understanding of the survey method used, it seems to have been sound.
The chief problem was what the investigater had to work with. He supposedly draw towns randomly, weighted by their population. Census data for towns in Iraq appear to be WAGs. Fallujah for example, is reported in news accounts of having 200,000 or 300,000 people, or any number in between. How can one properly draw towns, based upon such fuzzie population figures?
I’ve got two questions and a comment about this blog: 1) What is this guy’s background? 2) Can anyone point me to an explaination of the “confidence interval” that he refers to?
And my comment: his concluding statement that he’s “suspicious” of the results of the paper because the 100,000 figure “strikes him as high” seems to be the kind of thing I was hoping to avoid, but I appreciate the link, and read it very carefully.
Steve is, as best I can learn, an economist with political enthusiasms, and he has a very good grasp of statistics. Politically, he’s in the middle of the Right-Wing, with occasional Centrist lapses. ‘Confidence Interval’ is a range of possible outcomes, stated with a probability of confidence.
In short, when the Lancet publishes a Confidence Interval of 95% for a range of death of 8,000–194,000, it means that the authors of the study are 95% confident that somewhere between 8,000 and 194,000 civilians were killed, above and beyond the deaths expected in the usual run of events. Rather a large range, that.
That alone makes me suspicious, because that range is so large as to make any conclusions meaningless. Anyway, Steve’s statistical knowledge, and willingness to correct himself when he screws up, gives me very high confidence in his assessement of the study. I frequently don’t agree with his political positions, but I find him otherwise highly credible.
Anyway, his political slant is the reason I placed the disclaimer in my post above. I’d rather people went to the analysis with their antennae already primed and ready, rahter than they be surprised.