The Samovar


That Lancet report
October 19, 2006, 8:10 pm
Filed under: Politics

The Lancet medical journal recently published a study showing that the war in Iraq has caused approximately 655,000 excess deaths. This has understandably caused some extreme reactions, both in affirmation and denial.  Many people find this statistic difficult to believe, and I think this reaction is understandable. I am not going to attempt here a complete defence of the report and its conclusions, because I’m not an expert in these matters. However, I do want to put forward some observations.

  1.  The paper was published in a peer reviewed journal, which means it was subjected to very high academic and intellectual standards. The conclusions consequently cannot be just written off, as Bush has done, by saying that it’s “simply not credible”.
  2. The statistical analysis of the report claims that there is a 95% chance that the actual number of excess deaths is between about 390,000 and 940,000. The chance that it is less than 390,000 is therefore 2.5%, or one in 40. It’s possible, but quite unlikely. I’m not acquainted with the statistical model that they have used, but assuming that this confidence interval comes ultimately from a normal distribution, I estimate that this would mean that the chance that the true figure was 200,000 or lower would be 0.04% or about 1 in 2,200, and the chance that it was 100,000 or lower would be 0.002% or about 1 in 37,000. Daniel Davies wrote a good article explaining some of this in the Guardian.
  3. Most, but not all, of the denials of the conclusions of this report seem to be based on ignorance and rhetoric rather than any actual argument. This is of course only a heuristic way of judging the value of the report. A good example is Christopher Hitchens’ article in Slate.
  4. There are a few responses that I am aware of that merit some attention however. The Iraq Body Count website for example has issued a strongly critical press release. They do not attempt to address the statistical methods of the study, instead they attempt to show that if the conclusions of the report are correct, the consequences are at variance with other known facts. Personally, I find much of their reasoning a bit weak as it seems to rely on common sense ideas about how absurd such and such would be (and I don’t want to rely on my common sense understanding of what things are like in a war zone), but this is at least something that bears thinking about. For some discussion, see Leninology’s critique of IBC, and some of their responses to it.
  5. The Iraq Body Count counts deaths by a passive technique (they wait for information to be brought to them) rather than an active one (the Lancet authors actively sought information about deaths). Previous experience of estimates of numbers of people killed in war zones – according to the Lancet – show that passive techniques typically underestimate by at least 5 times and as much as 20 times. The passive estimate of the IBC is about 46,000 which if it were an underestimate by 5 times would become 230,000 or if it were an underestimate by 20 times would become  920,000. These numbers are similar in scale to the numbers in the Lancet report.
  6. Another interesting discussion can be found in an article by Gavyn Davies in today’s G2 section of the Guardian. He mentions one possible response which would lower the estimated number of excess deaths. The Lancet report estimates the pre-invasion death rate as 5.5 (which is also very close to the CIA World Factbook estimate), and the post-invasion death rate as 13.3. These death rates mean the number of people who die per year per 1,000 individuals, see the next point for some discussion of this. However, the United Nations apparently estimate the pre-invasion death rate to have been 9.5, which Davies says means that the Lancet estimate would mean the number of excess deaths was about 319,000 (he doesn’t give confidence intervals). Again, to come to a satisfactory conclusion about this would take a lot of work understanding the way that the Lancet / CIA / UN methods for estimating death rates worked.
  7. Death rate statistics are quite odd. I decided to look up the UK’s death rate to compare with the Lancet report estimates of the Iraq death rates and to my great surprise the UK death rate is approximately 10.1 according to the CIA World Factbook. On the face of it, it seems unlikely that Iraq’s pre-invasion rate would be as low as 5.5 if the UK’s is 10.1. This intuition seems to be unreliable though, as becomes clear by looking at a list of countries ranked by death rates. At the top of the scale are mostly African countries with rates in the 20s. At the bottom of the scale are many Middle Eastern countries. For example Kuwait and Brunei with rates of 2.4 and 2.6 respectively.

So what are we to make of all this? For most of us the details of the statistical and methodological techniques involved in these estimates are very difficult for us to make our own judgements about. Having said that, there doesn’t seem to be any serious criticism of the techniques involved and so it seems safe to assume they are sound. Possibly some bias has creeped in somewhere in some unclear way that the researchers overlooked, but it’s far from obvious. It seems unlikely on the basis of this that the report is a wild overestimate, and even if we consider that it might be half as much as they say it is (as Gavyn Davies’ article suggests might be the case), this means that enormous numbers of people have been killed because of this war. The whole debate has an element of quibbling about it, because even the very much lower figures of the Iraq Body Count website (44,000-48,000 at the time of writing) are shockingly high.

About these ads

6 Comments

When I was in the US last week, I was listening to the researchers themselves discuss their methodology and it seems that they have at least as much circumstantial evidence to back up their conclusions as the nay-sayers do. They recorded death certificates for everyone they polled, and even though the study is based on a statistically significant sample, we know that statistical samples, if big enough, are usually quite reliable.

Bush is in a horrendous situation. The Iraq experiment has failed abjectly, and each day brings yet more news that the immediate future of the country is hopeless and that American involvement there will soon come to an end. So, when he says that the numbers are “simply not creditable”, you have to understand where he is coming from. It’s his war, and coming to terms what is now obviously a massive failure does not seem to be built into his DNA. He also has to explain to the families of the 2,000 odd servicemen and women who have died there why he did it, not to mention all those who have been maimed or traumatised by their experiences there.

Comment by woodpigeon01

Excellent post

Comment by Chris

Thanks Chris.

woodpigeon, I completely agree. There’s also a good 10 minute discussion on the Lancet webpage with the authors of that report.

Comment by Dan Goodman

(I just deleted a comment that didn’t make any sense to me – I assume it was spam but if the original author wants it back just send me an email or post the comment again.)

Comment by Dan Goodman

[…] Dan has some more observations on the report. ———— This is a guest post.   |   Trackback link   | […]

Pingback by Pickled Politics » The death of an honest government

[…] Holland and Michael Schwartz, authors of  the Lancet report of October 2006, asked Iraqis the cause of death of their family members.  Of deaths for which families were […]

Pingback by Over 1,000,000,000 dead Iraqis from 2003 invasion « Craig Considine




Comments are closed.



Follow

Get every new post delivered to your Inbox.

%d bloggers like this: