«THE ‘SCIENCE’ OF MISCARRIAGES OF JUSTICE GARY EDMOND* I FORENSICS ON TRIAL Many, perhaps most, wrongful convictions involve forensic scientific ...»
376 UNSW Law Journal Volume 37(1)
THE ‘SCIENCE’ OF MISCARRIAGES OF JUSTICE
I FORENSICS ON TRIAL
Many, perhaps most, wrongful convictions involve forensic scientific and
medical evidence that was exaggerated, misleading or simply mistaken.1 Many
ordinary criminal proceedings are substantially unfair, some are no doubt miscarriages of justice, because forensic science and medicine evidence that is not known to be reliable is admitted and relied upon. For a variety of reasons serious limitations with expert evidence seem to be infrequently identified, explained or conveyed. Those responsible for admissibility, evaluating the evidence, or reviewing the evidence and the safety of the conviction on appeal, are not necessarily apprised of serious limitations and methodological heterodoxy. This is an appalling vista.2 This article endeavours to provide some explanation for why so much unreliable, speculative and weak expert evidence is relied upon in criminal proceedings and how we might refine our practices to reduce the volume of such evidence and, simultaneously, the number of miscarriages of justice. This article suggests that among the most useful reforms that could be imposed is the imposition of a reliability standard to regulate the admission and enhance the evaluation of incriminating expert opinion evidence.3 This article draws on the experience of past miscarriages of justice, royal commissions and criminal case review commissions, the experience of innocence projects, as well as extensive commentary on forensic science and medicine * Professor, Australian Research Council Future Fellow, and Director, Expertise, Evidence and Law
Program, Faculty of Law, The University of New South Wales, Sydney 2052, Australia, email:
email@example.com; Professor, School of Law, Northumbria University; and Visiting Professor, Peter Wall Institute of Advanced Studies, University of British Columbia. This research was supported by the Australian Research Council (FT0992041 and LP120100063). Thanks to Emma Cunliffe, Mehera San Roque, David Hamer, Andrew Ligertwood and referees for their comments.
1 See, eg, Morgan v The Queen (2011) 215 A Crim R 33 (‘Morgan’); Wood v The Queen  NSWCCA 21 (‘Wood’); Gilham v The Queen (2012) 224 A Crim R 22 (‘Gilham’).
2 Cf McIlkenny v Chief Constable of the West Midlands  1 QB 283.
3 For reasons that will become obvious, this article focuses exclusively on expert evidence adduced by prosecutors (ie, on behalf of the Crown or state). When speaking about reliability this essay is referring to the trustworthiness of evidence. ‘Reliability’ also possesses a technical, or scientific, meaning that is far more constrained.
evidence developed over several decades.4 It also draws on my own experience observing trials and appeals, reading expert reports, transcripts and judgments across several domestic and foreign jurisdictions, as well as my participation on a national standards committee. 5 While these insights are important, they supplement a confronting recent development: reviews and reports by peak scientific and technical bodies. Over the last five years there have been several important inquiries into the forensic sciences. The resulting reports characterise ‘law’s greatest dilemma in its heavy reliance on forensic evidence’ as ‘the question of whether – and to what extent – there is science in any given forensic science discipline.’ 6 Here, in the context of a discussion of forensic science evidence and miscarriages of justice, it is my intention to raise two closely related issues that are central to the question of the need for a reliability-based admissibility standard.
The first is that many of the forensic sciences do not appear to be demonstrably reliable. That is, we do not know whether many of the techniques used in criminal investigations and relied upon in criminal prosecutions actually work (and, compounding things, some of the reliable techniques are not always performed in ways that are conducive to reliable outcomes). These disturbing findings are briefly discussed in Part II. Secondly, Australian lawyers and judges did not, in some kind of endogenous manner, recognise or develop their procedures and jurisprudence in ways that identified or provided credible responses to the concerns expressed in the recent reports. To put this more 4 United Kingdom, Report into the Circumstances Surrounding the Conviction Arising out of the Bomb Attacks in Guildford and Woolwich in 1974, Final Report (1994); United Kingdom, Royal Commission on Criminal Justice, Report (1993); Northern Territory, Royal Commission of Inquiry into the Chamberlain Convictions, Report of the Commissioner (1987); Manitoba, Commission of Inquiry into Certain Aspects of the Trial and Conviction of James Driskell, Report (2007); Ontario, Commission on Proceedings Involving Guy Paul Morin, Report (1998); R v Truscott (2007) 50 CR (6th) 1 (Ontario Court of Appeal).
For critical commentary, see Clive Walker and Keir Starmer (eds), Miscarriages of Justice: A Review of Justice in Error (Oxford University Press, 1999); Richard Nobles and David Schiff, Understanding Miscarriages of Justice: Law, the Media and the Inevitability of a Crisis (Oxford University Press, 2000); Michael Naughton, Rethinking Miscarriages of Justice: Beyond the Tip of the Iceberg (Palgrave Macmillan, 2012); Brandon L Garrett, Convicting the Innocent: Where Criminal Prosecutions Go Wrong (Harvard University Press, 2011); Michael J Saks and Jonathan J Koehler, ‘The Coming Paradigm Shift in Forensic Identification Science’ (2005) 309 Science 892; Margaret A Berger, ‘What Has a Decade of Daubert Wrought?’ (2005) 95 American Journal of Public Health 59; Peter J Neufeld, ‘The (Near) Irrelevance of Daubert to Criminal Justice and Some Suggestions for Reform’ (2005) 95 American Journal of Public Health 107; Erica Beecher-Monas, Evaluating Scientific Evidence:
An Interdisciplinary Framework for Intellectual Due Process (Cambridge University Press, 2007).
5 See, eg, Gary Edmond et al, ‘Admissibility Compared: The Reception of Incriminating Expert Evidence (ie, Forensic Science) In Four Adversarial Jurisdictions’ (2013) 3 University of Denver Criminal Law Review 31; Gary Edmond and Joelle Vuille, ‘Comparing the Use of Forensic Science Evidence in Australia, Switzerland and the United States: Transcending the Adversarial/Non-adversarial Dichotomy’ (2014) forthcoming Jurimetrics Journal. The author is a member of the Forensic Analysis CH-041 Committee of Standards Australia.
6 Committee on Identifying the Needs of the Forensic Science Community, ‘Strengthening Forensic Science in the United States: A Path Forward’ (Report, National Academy of Sciences, 2009) 9 (emphasis in original) (‘NAS Report’).
378 UNSW Law Journal Volume 37(1) starkly, our trial system – with its oft-valorised values, rules, safeguards and experienced personnel – neither recognised nor exposed fundamental and widespread problems with many techniques and a great deal of forensic science evidence. Rather, persisting with longstanding and antiquated approaches to expertise and fairness, our courts have tended to discount or overlook serious problems. Trial judges have been overly accommodating in the admission of incriminating expert evidence and appellate courts have, in general, been insensitive to problems and dangers with a great deal of contemporary forensic science and forensic medicine. The failure to recognise endemic limitations and react systematically has, ironically, contributed to the continuing, and perhaps expanding, use of unreliable, weak and speculative incriminating expert evidence. Simultaneously, it has rendered courts oblivious to the depth and prevalence of evidentiary problems, the erosion of the standard of proof and, quite likely, the prevalence of miscarriages of justice.
II WHAT DO WE NOW KNOW ABOUT THE FORENSICSCIENCES?
To support the need for a reliability standard, this article draws on three recent reports that provide unprecedented insight into the condition of the forensic sciences. The most important, most wide-ranging and most critical of these reports was based on an inquiry by a committee of the United States (‘US’) National Academy of Sciences (‘NAS’). Over two years, this formidable multidisciplinary committee received submissions, held hearings and made inquiries. The result of that process was the NAS Report.7 The Report captures the committee’s surprise at what it uncovered: many of the forensic sciences appear to lack scientific foundations. The Report is, in response, highly critical of a great deal of contemporary forensic science (and medicine) evidence and practice. What follows draws heavily on this seminal report, particularly the findings and recommendations – see Boxes 1 and 2.
Box 1: NAS Report – Findings
Validation of techniques The simple reality is that the interpretation of forensic evidence is not always based on scientific studies to determine its validity. This is a serious problem. … there is a notable dearth of peer-reviewed, published studies establishing the scientific bases and validity of many forensic methods.8 Little rigorous systematic research has been done to validate the basic premises and techniques … The committee sees no evident reason why conducting such research is not feasible …9
Standards Often there are no standard protocols governing forensic practice in a given discipline. And, even when protocols are in place … they often are vague and not enforced in any meaningful way. … These shortcomings obviously pose a continuing and serious threat to the quality and credibility of forensic science practice.10 Accuracy and uncertainty Few forensic science methods have developed adequate measures of the accuracy of inferences made by forensic scientists. All results for every forensic science method should indicate the uncertainty in the measurements that are made, and studies must be conducted that enable the estimation of those values.11 Terminology and expression [M]any terms are used by forensic examiners in reports and in court testimony … Such terms include … ‘match’, ‘consistent with’, ‘identical’, ‘similar in all respects tested’, and ‘cannot be excluded as the source of.’ … the forensic science disciplines have not reached agreement or consensus on the precise meaning of any of these terms. This imprecision in vocabulary stems in part from the paucity of research … and the corresponding limitations in interpreting the results of forensic analyses.12 Contextual bias (and human factors) Some initial and striking research has uncovered the effects of some [cognitive and contextual] biases in forensic science procedures, but much more must be done to understand the sources of bias and to develop countermeasures. … The forensic science disciplines are just beginning to become aware of contextual bias and the dangers it poses. The traps created by such biases can be very subtle, and typically one is not aware that his or her judgment is being affected.13 Following the inauguration of the NAS inquiry, the US National Institute of Standards and Technology (‘NIST’) and the National Institute of Justice (‘NIJ’) jointly sponsored a review of latent fingerprint evidence, particularly the role of ‘human factors’. 14 A large multidisciplinary committee, the Expert Working Group on Human Factors produced a report that identified many areas of practice that require attention and reform. At about the same time that the NIST/NIJ review was established in the US, a senior Scottish judge was conducting a public inquiry into fingerprint evidence in Scotland.15 Endorsing and affirming many of the findings and recommendations made by the NAS committee, these reports were critical of the way a potentially valuable technique had been processed and reported in ways that were (and remain) insensitive to limitations and inattentive to a range of substantial risks.
10 Ibid 6.
11 Ibid 184.
12 Ibid 185–6.
13 Ibid 184–5.
14 Expert Working Group on Human Factors in Latent Print Analysis, ‘Latent Print Examination and Human Factors: Improving the Practice through a Systems Approach’ (Report, US National Institute of Standards and Technology and US National Institute of Justice, 17 February 2012) (‘NIST/NIJ Report’).
15 Anthony Campbell, ‘The Fingerprint Inquiry Report’ (Report, The Fingerprint Inquiry Scotland, December 2011) vol 1 (‘Scottish Fingerprint Inquiry Report’).
380 UNSW Law Journal Volume 37(1) What did the reviews uncover that so unsettled the scientists, engineers, statisticians and judges involved?16 In summary, the reports concluded that many forensic science techniques in routine use had not been validated. This means that we do not know whether the techniques actually work and cannot be confident about the scope of any claimed abilities. 17 Moreover, standards are often vague, unenforced or not derived from empirical studies. 18 In addition, relatively few forensic scientists are shielded from domain irrelevant information.
This is significant because several prominent experimental studies have confirmed the corrosive impact of contextual influences (eg, suggestion) on human practices, particularly interpretation – see ‘Contextual bias’ in Boxes 1, 2, and 3.19 The implication from the NAS and other reports is that many forensic science and medicine techniques are of unknown value. In many cases we do not know if analysts can actually do the things that courts routinely enable them to testify about. And, if they can do them, we do not have a clear idea about how accurate they are, or the limits of any abilities. This led to a series of substantial recommendations for reform – see Box 2.
Box 2: NAS Report – Recommendations