BMJ 2000;320:2-3 ( 1 January ) Editorials
Doctors and lawyers should get probability theory right
In a recent case of DNA evidence the probability of a chance match was quoted as 20 million to one. The accurate statementthat the defendant or two other unknown people in the United Kingdom could have committed the offenceis much less impressive. Other evidence was overwhelming, but this may not always be true, especially with matches from DNA databases. Even more problematic than the issue of presenting statistical evidence fairly is the problem of getting it wrong.
On 9 November at Chester Crown Court Sally Clark, a Cheshire solicitor, was convicted, by 10-2 majority, of smothering her two infant children. With conflicting forensic evidence, the Crown's case was bolstered by an eminent paediatrician testifying that the chances of two cot deaths happening in this family was vanishingly small 1 in 73 million. This seriously misunderstands probability theory. It is speculation whether Sally Clark would have been acquitted without this evidence. But with this mathematical error prominent the conviction is unsafe.
Imagine an archery target with two arrows sticking in
the very centre of it. This provides greater evidence of the skill of the
archer if the target was in place before the arrows were fired than if it
was drawn around them afterwards. Probability theory requires calculation
of the probability not only of the event in question but also of all
events that are as extreme or more extreme. When the target is drawn first
you calculate the chance of both arrows hitting the centre of the target.
But when the target is drawn round the arrows afterwards you calculate the
chance of both arrows hitting the same point, whatever that point. With
two independent arrows one probability is the square of the other.
Suspicion was drawn to Sally Clark by the occurrence
of two deaths so the probabilities should not have been squared. The odds
of 1 in 73 million shrink to 1 in 8500. But this figure is itself
meaningless. There is in fact a wall full of arrows with the target drawn
around the two that are close together and the others ignored.
Mathematical formulas for this situation often surprise people. For
example, with only 23 people in a room the odds are better than 50% that
two of them have the same birthday.
From whole population data Reese calculates the square of the population risk of cot death as 1 in 2.75 million.1 There are 378 000 second or subsequent births each year in England. So if cot deaths are random events two cot deaths will occur in the same family somewhere in England once every seven years. But cot deaths are not random events. There have been several studies of recurrence. At least one study did show no increase in recurrence rates.2
But several others showed recurrence rates about five
times the general rate,3-5 implying recurrence somewhere in England about
once every year and a half. Two studies showed even higher rates. 6
The fact that studies of recurrence have been done means this event is not vanishingly rare. In a case series of recurrent infant death Emery classified two cases as recurrent cot death out of 12 cases occurring in Sheffield in 20 years.8 Wolkind et al found five cases in their unsystematic English case series of 57 recurrent infant deaths.9
Both these studies distinguished cot death from
accident, illness, murder, and neglect. The prosecution used the figure of
1 in 73 million rather than 1 in 2.75 million because of the family's
affluence. Yet taking data from an epidemiological group and applying it
stereotypically to all members is an example of the ecological fallacy.
Social class is a complex reality of interassociated
circumstanceseducation, work, income, lifestyle, culture, contacts,
residence, opportunities, social class of origin, etcstatistically
summarised for use in population studies by selecting the one variable
which performs best as an indicator. This does not mean that individuals
have the attributes of the statistical group.
Guidelines for using probability theory in criminal
cases are urgently needed. The basic principles are not difficult to
understand, and judges could be trained to recognise and rule out the kind
of misunderstanding that arose in this case. Never again must mathematical
error be allowed to conflict with mathematical fact as if each were a
legitimate expert view.
What is our profession's responsibility for the quality of expert evidence given by doctors? Medical evidence is trusted, and we must retain that situation and ensure that it is not abused. It is possible to be an extremely good doctor without being numerate, and not every eminent clinician is best placed to give epidemiological evidence.
Doctors should not use techniques before they have
acquainted themselves with the principles underlying them.When errors
occur we expect them to be admitted, learnt from, and corrected. Should
clinical governance extend to the courtroom? Expert witnesses can hold a
substantial part of defendants' lives in their hands. Defendants deserve
the same protection as patients.
Stephen J Watkins, director of public health.
Predisposing biases: SJW is a vice president and
immediate past president of the Medical Practitioners' Union, which is
predisposed to support the civil liberties movement. He has no personal
acquaintance with people involved in this case.
1. Reese A. In Statistics and justice. www.stats.gla.ac.uk/allstat/. Accessed November 1999.
2. Peterson DR, Subotta EE, Dubing JR. Infant mortality among subsequent siblings of infants who died of sudden infant death syndrome. J Pediatr 1986; 108: 911-914[Medline].
3. Oyen N, Skjaerven R, Jurgens LM. Population-based recurrence risk of sudden infant death syndrome compared with other infant and foetal deaths. Am J Epidemiol 1996; 144: 300-305[Medline].
4. Guntheroth VG, Lohmann R, Spiers PS. Risk of sudden infant death syndrome in subsequent siblings. J Pediatr 1990; 116: 520-524[Medline].
5. Irgens LM, Skjaerven R, Peterson DR. Prospective assessment of recurrence risk in sudden infant death syndrome siblings. J Pediatr 1984; 104: 349-351[Medline].
6. Froggart P, Lynas MA, McKenzie G. Epidemiology of sudden unexpected death in infants ("cot death") in Northern Ireland 1971. Br J Soc Prev Med 1984; 25: 119-134.
7. Beal SM, Blundell HK. Recurrence incidence of sudden infant death syndrome. Arch Dis Child 1988; 63: 924-930[Abstract].
8. Emery JL. Families in which two or more cot deaths have occurred. Lancet 1986; i: 313-315.
9. Wolkind S, Taylor EM, Waite AJ, Dalton M, Emery JL. Recurrence of unexpected infant death. Acta Paediatrica 1993; 82: 873-876[Medline].
Mothers Against MSBP Allegations
Heart-To-Heart Discussion Group