A call for more science in forensic science

• forensic science
• justice
• DNA
• crime
• investigation

Forensic science is at a crossroads. It is torn between the practices of science, which require empirical demonstration of the validity and accuracy of methods, and the practices of law, which accept methods based on historical precedent even if they have never been subjected to meaningful empirical validation. The field is in dire need of deep and meaningful attention from the broader scientific community. Without such guidance, forensic science and law enforcement risk withholding justice from both defendants and crime victims. The scientific community must step forward to promote, defend, and advocate for science in forensic science.

The issue is of particular importance in light of the decision by the Department of Justice (DOJ) in April 2017 to terminate the National Commission on Forensic Science (NCFS), a group (on which we served) that was charged with advising the federal government on improving the parlous state of the forensic science. Remarkably, the DOJ took this step despite recent reports from the National Academy of Science (NAS) and the President’s Council of Advisors on Science and Technology (PCAST) that highlighted many problems, including the fact that some forensic methods have never been validated. Some of these methods are clearly invalid. The most egregious case is bite mark identification, which has been discredited by both scientific studies and false convictions based on the method. However, bite marks continue to be accepted in United States courts as a matter of precedent: that is, not because they are valid but because they were accepted in the past. As science—and forensic science more specifically—continues to advance, it becomes increasingly absurd to ask or expect lawyers, judges, and juries to take sole responsibility for critically evaluating the quality and validity of scientific evidence and testimony.

The structure of the field of forensic science inhibits vital reforms. Almost all publicly funded laboratories, whether federal, state, or local, are associated with law enforcement. At the very least, this creates an inherent conflict-of-interest and leads to legitimate concerns of objectivity and bias. The linkage of forensic laboratories with prosecutorial entities dates back as far as 13th century China, was pervasive in Europe in the mid-late 19th century, and spread from there to the United States (1⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓–14).

Some forensic methods have been rooted in science. Medicolegal death investigation emerged from medical science, because death investigation was connected to the protection of public health. Techniques of analytical chemistry were applied to the certain types of evidence, such as seized drug analysis, toxicological analysis, and aspects of instrumental analysis applied to trace evidence. More recently, molecular biology gave rise to DNA typing to forensic applications.

The evolution of other forensic disciplines, particularly those related to pattern evidence, followed a different course, having been developed primarily within law enforcement environments or at the behest of law enforcement. Disciplines, such as fingerprints, firearms, and tool marks, blood-stain pattern analysis, tread-impression analysis, and bite mark analysis matured largely outside of the traditional scientific community during a time when admissibility standards for scientific evidence had yet to be formulated. Thus, admissibility of such evidence rightly or wrongly created judicial precedent in decisions that often did not—or could not—involve the level of research that would today be needed to establish scientific validity.

The adaptation of DNA typing methods to forensic casework, a pivotal event in forensic science, catalyzed a reassessment of the scientific validity of other methods used in forensics. In the 1980s, Alec Jeffreys of the University of Leicester discovered that segments of repetitive DNA were tremendously variable among individuals and coined the term “DNA fingerprinting” (15). The rapid embrace of DNA typing, beginning in the late 1980s and continuing through the turn of the century, had far-reaching implications in the judicial system. The probabilistic nature of DNA evidence and its acceptance by the courts also played a role in shaping modern views on scientific validity. Before DNA typing, analysis of blood evidence relied on ABO blood group and secretor status, which could afford population frequencies on the order of n-in-100. DNA typing allowed a person to be linked to a sample with frequencies of less than one across the population of the world (i.e., less than one in eight billion). The use of rigorously estimated probabilities as a tool to weigh the relative importance of the data marked a critical turning point in forensic science.

During the same time, fingerprint analysis was also used to identify individuals as the source of impressions, but without either population data (on the similarity among fingerprints) or empirical studies (on the performance of examiners) providing estimates of the probability for false-positive matches. In retrospect, it is clear that DNA evidence and its success changed our views and expectations of forensic science.

In the 1990s, three critical Supreme Court rulings in civil cases provided guidance regarding the admissibility of evidence in federal cases. In Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 US 579 (1993), the judge was assigned a gatekeeping role to ensure that expert scientific testimony was found to be reliable before it could be admitted as evidence. In General Electric Co. v. Joiner, 522 US 136 (1997), the Court made clear that scientific testimony must be relevant to the case at hand to be admissible. Finally, the decision in Kumho Tire Co. v. Carmichael, 526 US 137 (1999) broadened the scope of expert testimony to include all types of technical evidence, while holding fast to the reliability and relevancy requirements. These three cases, often referred to as the Daubert trilogy, generated a two-pronged test for the admissibility of evidence ruling, namely that scientific evidence used in court must be both reliable and relevant.

The Daubert trilogy represents a critical milestone in the intersection of science and the law by demanding that admissibility decisions rely on contemporaneous scientific standards. Although the admissibility of DNA evidence slightly preceded the Daubert trilogy, it provides a good model for how modern scientific advances should be integrated into the justice system: namely, scientific validation should precede admissibility.

Additionally, DNA typing has had a significant impact on forensic science through exonerations of false convictions. As noted in a recent summary report (16), for convictions in the 1974–2016 period, DNA evidence has overturned more than 100 false convictions. Causes of false convictions are mistaken witness identification, perjury or false accusations, false confessions, official misconduct, inadequate legal defense, and false or misleading forensic evidence. In those cases where forensic science was cited as a primary cause of the false conviction, the most common methods used were forensic biology (serology), hair examinations, and bite marks.

This does not mean that all previously admitted types of evidence are necessarily invalid, but it does require, at the very least, that validity be now established by appropriate scientific standards before they can continue to be used. This requirement poses a dilemma to prosecutors—and to some extent to law enforcement—who face an inherent risk and disincentive in arguing for scientific validation studies that could call into question past convictions based on methods that no longer pass muster. Even when scientific studies clearly debunk a methodology, some prosecutors appeal to past legal precedent to persuade courts to admit evidence, as seen in the case of bite mark evidence. The scientific community must step up to counter this pressure.

The NAS has been at the forefront of these efforts since the early 2000s (17). In November 2005, the Science, State, Justice, Commerce, and Related Agencies Appropriations Act of 2006 called upon the National Research Council (NRC) to conduct a study of forensic science. The exhaustive study resulted in the 2009 publication of Strengthening Forensic Science in the United States: A Path Forward (18), which concluded that “with the exception of nuclear DNA analysis…no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.” The 2009 report (18) recommended the creation of a “new, strong, and independent entity that could take on the tasks that would be assigned to it in a manner that is as objective and free of bias as possible—one with no ties to the past and with the authority and resources to implement a fresh agenda designed to address the problems found by the committee and discussed in this report.” Notably, the NRC report was unambiguous that this entity be outside of the jurisdiction or control of the DOJ.

Rather than establishing such an independent entity, the government created the NCFS, which was established by the DOJ in partnership with the National Institute of Standards and Technology (NIST). The NCFS functioned from 2013 to 2017, during which time it held 13 meetings. It was a diverse body composed of representatives of several stakeholder communities, including forensic scientists, law enforcement, judges, attorneys, and independent scientists not associated with forensic science. The 49 commissioners served over two terms, heard presentations from 140 invited presenters, and approved 43 documents and summary reports. Given its heterogeneous composition and expertise, the NCFS took time to function efficiently. Only one document was approved before its fifth meeting, compared with eight at the September 2016 meeting alone. This timeline shows evidence of the learning curve commissioners were on as they began a deliberative process to achieve consensus on reports and summary documents.

As examples, the NCFS recommended the creation of postdoctoral training programs in forensic science to encourage the emergence of an inquisitive and investigative scientific culture, which the National Institute of Justice (part of the DOJ) quickly embraced. One practical recommendation was the abandonment of language the Commission found to be meaningless and misleading, such as claims by experts that their conclusions were correct to a “reasonable scientific certainty.” Of significance was the commission’s recommendation that forensic techniques be subjected to independent validation before being introduced into common use and that the NIST should be responsible for such oversight. Beyond its recommendations, the NCFS provided a first-ever national-level venue for communication and understanding among the many disciplines represented. During NCFS discussions, it became clear that the scientific and legal communities often had different interpretations of what constituted “error” in forensic analysis, with the former recognizing error as an intrinsic aspect of any measurement process and the latter often viewing error as synonymous with a mistake: that is, the inappropriate application of a procedure or technology. Although NCFS recommendations do not have the force of law, the fact that they emerged from a commission composed of such different stakeholders gave them moral force. Unfortunately, all of the hard work needed to forge such a heterogeneous group into a body that had learned to reach consensus was lost when the DOJ declined to renew the NCFS in early 2017.

In late 2016, a PCAST report (19, 20) highlighted why bodies like the NCFS are needed. PCAST based its conclusions on a review of more than 2,000 papers in the forensic science literature, as well as interviews with forensic scientists and stakeholders in the legal community. The report identified two gaps requiring attention: (i) a need for clarity about the scientific standards required to establish the validity and reliability of forensic methods, as well as to measure their accuracy; and (ii) a need to scientifically establish the validity and reliability of particular forensic methods that had never been properly validated. Providing an independent confirmation of many of the findings of the prior 2009 NRC report, the PCAST report concluded that empirical testing is not merely one among various alternative ways to establish scientific validity; rather, it is the only scientific basis for doing so. Furthermore, the PCAST report established that, in the 7 y since the 2009 NRC report, little progress had been made to address the criticisms raised in that report. The sole exception was latent-fingerprint analysis, which had been subjected to validity testing. A key issue is how to extend this one example to other forensic methods.

After terminating the NCFS, in April 2017, the DOJ proposed opening a new office for forensic science within the department and named a prosecutor to lead this effort. This new proposal is highly problematic. Specifically, it goes against the recommendations of the 2009 PCAST report, which strongly suggested that the DOJ not be involved in evaluating the use of forensic science. Although the NCFS was not entirely independent, it did include some independent stakeholders: scientists outside the realm of forensic science. Putting a prosecutor in charge of forensic science perpetuates an irreconcilable conflict-of-interest and reinforces the dominance of the prosecutorial perspective. Prosecution entities, by the nature of our adversarial legal system, have little incentive to embrace scientific advances that could risk undermining past convictions and current prosecutions. Conversely, defense entities have incentives to constantly question and raise doubts regarding scientific results that do not support their desired outcome. The role of prosecutors and defense attorneys is to win cases through competing arguments (i.e., the adversarial system). Neither “side” can or should be expected to evaluate scientific integrity on its own merits. The need for an independent and dedicated champion of forensic science has never been clearer.

The limitations of some forensic science methods have been exposed, often by forensic scientists themselves. The larger scientific community must now come to the aid of our forensic colleagues in advocating both for: (i) the research and financial support that is so clearly needed to advance the field and (ii) the requirement for empirical testing that is so clearly needed to advance the cause of justice. Vocal and continual advocacy for scientific independence is needed, along with policy recommendations and a concerted effort to ensure that this issue stays in the public conscience. Independent review efforts should be launched and supported. Forensic scientists have long complained that their work is not always valued by their scientific colleagues because of its applied nature; it is time for the scientific community to move beyond that conceit. Research and academic scientists should become educated about forensic science and take active steps to welcome the discipline into the larger scientific community. A broad effort can help illuminate the causes of failures, help predict when failure is likely to occur, and aid in the development of strategies to mitigate or circumvent those conditions. Because it represents the wide gamut of scientific disciplines that are essential to forensic science, the NAS remains in a prime position to continue the dialogue between the academic and forensic science communities. If we are unwilling to confront the issue of accuracy in our justice system, what cause is worthy?