Fallacies of forensic science

Steven Barnes was exonerated of murder and rape charges 20 years into his sentence when the Innocence Project proved evidence against him was wrong.

The Innocence Project in the US has helped free 230 wrongly convicted people, including 17 on death row, largely by proving the evidence incorrect. Widely used forensic techniques have unfortunately high error rates. The conviction of Steven Barnes for the rape and murder of a woman in Whitestown, New York, in 1985, seemed like an open-and-shut case. Eye-witnesses testified that they had seen the 24-year-old in town on the night of the attack, and thought they may even have seen Barnes with his victim. Then came the forensic evidence, with the jury being told that the soil on Barnes' lorry tyres was similar to that at the crime scene. Most compelling of all was the revelation that a fabric pattern found on Barnes's lorry matched that on the victim's jeans.

In the face of such a wealth of evidence, Barnes was found guilty and sentenced to 25 years to life. A few weeks ago, he was released after serving almost 20 years - not on parole, but as a free man exonerated of all guilt. Barnes had become the latest triumph for the work of the Innocence Project, set up in 1992 at the Cardozo School of Law, New York, to overturn wrongful convictions. Over the years, the Innocence Project has helped free over 230 innocent people, including 17 who faced execution. Time and again, its researchers have revealed the unreliability of forensic evidence, all too often deemed infallible by judges, juries, and the scientists who wield it. Now their calls for action have been taken up by the prestigious US National Academy of Sciences, in a hard-hitting report set to become a watershed in the use of forensic techniques worldwide.

The report reviews such familiar sources of evidence as fingerprints and hair matching, and shows that the term "forensic science" is frequently an oxymoron. It focuses particular attention on a critical feature of forensic techniques: their error rates. We have all seen it in the movies: a forensic scientist telling a jury about evidence that matches something found on the accused; a hair sample, say, or a fingerprint. It sounds pretty damning - until one asks what, exactly, does "match" mean? Outside the movies, forensic evidence is rarely clear cut, and the idea of a perfect match is a fiction. Instead, all such evidence has some probability of being misleading, in other words, an error rate. Or, to be more precise, error rates, as every source of evidence can mislead in two ways.

The most familiar is the so-called false positive - that is, evidence that wrongly identifies someone as the culprit. But as the authors of the NAS report stress, this is only half the story: evidence can also produce false negatives, and exonerate the real culprit. To be truly compelling, the false positive and false negative error rates of the source of evidence must be both known, and low. DNA profiling owes its power to the fact that both these error rates have been measured, and shown to be exceptionally low. The false positive error rate is typically much less than one in a million, while the chances of a false negative - failing to get a match between a crime scene sample and the true culprit - are similarly low.

Disturbingly, however, the same cannot be said about many widely-used forensic techniques. As the authors of the NAS report point out, their errors have never been established - making their evidential worth unknown. The authors state bluntly that the clarity over error rates is "absolutely critical" for forensic science, their measurement being "key components of the mission of forensic science".

Steve Barnes' wrongful conviction shows what can go wrong when juries trust unproven sources of evidence. Eyewitness testimony is notoriously unreliable: according to the Innocence Project, misidentification may be the single greatest cause of wrongful convictions in the US, playing a role in more than 75 per cent of convictions overturned through DNA testing. In the Barnes case, the eyewitness testimony was questionable at best. Seemingly far more impressive were the "matches" between the soil sample and the fabric patterns linking Barnes to his supposed victim. In reality, the error rates for these techniques have never been established - leaving a huge question-mark over their evidential weight. As the Innocence Project lawyers stated at the time of his release: "It is impossible to know how many other soil samples might be similar to soil from the crime scene or the likelihood that other jeans have the same pattern". Advances in DNA testing exonerated Barnes, excluding him from being the culprit in the 1985 murder.

No forensic technique rivals DNA profiling for proven evidential weight. Yet not even its impressively low error rates can protect it from that ineluctable source of error: humans. Ultimately, DNA profiling is only as good as those who carry it out, and they do not always get it right. Take the case of Raymond Easton, arrested in 1999 for a burglary in Bolton, England. Forensic scientists had compared samples taken from the crime scene with those held on the UK National DNA Database - and found a match with Eason, whose DNA was on the database following a domestic fracas. Easton was told that the DNA evidence against him was compelling, with the chance of getting so good a match from someone innocent being just one in 30 million.

Guilty as charged? Not quite: Easton lived over 300km from the crime scene, and suffered from Parkinson's Disease so severely that he could not dress himself, let alone commit a burglary. This prompted Easton's lawyer to do the unthinkable, and challenge the DNA evidence. Sure enough, further tests showed the original match was faulty, and Easton was released. Similar cases have emerged over the years; what makes them especially worrying is that only their sheer implausibility prevented them from reaching court. Innocent people without a rock-solid alibi can only hope to convince a jury that forensic evidence is far from infallible. And as the bulging files of the Innocence Project show, there is little hope of that.

Robert Matthews is Visiting Reader in Science at Aston University, Birmingham, England