About a decade ago, the tools of molecular biology that had unraveled some of the mysteries of heredity began to be applied to criminal investigations and trials. Law enforcement agencies committed substantial resources to the technology, and prosecutors, defense counsel, and judges struggled with the terminology and ideas of modern genetics. But differences of scientific opinion and legal maneuvering intensified to the point where commentators began to speak of "war"1and "free fire zones."2
In 1989, before the conflict had escalated, the National Academy of Sciences (NAS), acting through the National Research Council (NRC),3formed a Committee on DNA Technology in Forensic Science. This committee was composed of geneticists, forensic scientists, bioethicists, a law professor, and a judge.4In April 1992, it issued a thoughtful report (NRC I) containing a variety of recommendations on laboratory practices and regulation, development and presentation of evidence, and privacy.5
Strangely, NRC I did not resolve many controversies. Indeed, the report itself became the target of bitter criticism from population geneticists and statisticians, largely because of its endorsement of the "ceiling principle" for estimating the probability of finding a DNA profile in a randomly selected individual. That method was attacked for being neither a "principle" nor a "ceiling," but an ad hoc and arbitrary procedure.6And in truth, the method cannot be defended on purely scientific grounds. It was a policy-driven compromise intended to permit DNA evidence to be admissible and useful; it skirted rather than addressed the merits of a scientific debate over the effect of "population substructure" on the relative frequencies of DNA profiles in the general population.7
Moreover, the 1992 report distressed certain prosecutors and other law enforcement personnel. They saw courts using NRC I to exclude DNA evidence and a few experts arriving at absurd results by manipulating selected aspects of the recommendations about computing genotype frequencies.8
In April 1993, one year after NRC I, the director of the FBI asked the NAS to conduct a rapid follow-up study to resolve the questions of population genetics that the committee had tried to finesse with the ceiling principle.9After studying the request10and securing funding,11the NRC appointed a second committee late in 1994.12This committee differed from the first in several respects. For one, its task was highly circumscribed--"[t]he committee was formed to update and clarify discussion of the principles of population genetics and statistics as they apply to DNA evidence."13For another, it took its mission to be limited to informing the legal and scientific communities of what procedures could be justified under current scientific knowledge.14By and large, it made no recommendations to courts on the constraints that should exist on the admissibility of DNA evidence or to legislatures on the regulatory scheme that should be imposed on forensic laboratories. "We take no sides on such legal debates," the committee wrote in its 1996 report (NRC II).15 Instead, the committee was content to "trust that our efforts to explain the state of forensic science and some of the social science findings that are pertinent to resolving these issues will contribute to better-informed judgments by courts and legislatures."16
The second committee made recommendations17in three general areas: laboratory procedures, statistical assessments of laboratory results, and social science research. Just as NRC I did,18NRC II calls for high quality standards, accreditation, regular proficiency testing, and early sample splitting to permit independent testing.19
The "central question" for the second committee, however, was not how to run or regulate a laboratory, or how to decide whether two DNA samples match. It was "[w]hat information can a forensic scientist, population geneticist, or statistician provide to assist a judge or jury in drawing inferences from the finding of a match?"20NRC II devotes several chapters to an exposition of population genetics models and statistical issues and arrives at a series of recommendations for computing the probability that two samples would have the same genotype under various hypotheses: that the samples come from different, unrelated individuals in a large population; that they come from the same subgroup in the population; and that they come from close relatives.21NRC II provides formulas for estimating these probabilities and discusses the circumstances under which each should apply.22In this area, the committee parts company with its predecessor, which favored a single "ceiling" figure for all situations.23NRC II also recommends modifying the "random-match probability" when a match arises not from a single comparison of two samples, but from multiple comparisons in a search through a database of offenders.24
But the report looks forward to a time when "systems [will be] so powerful that no statistical and population analyses will be needed, and (except possibly for close relatives) each person in a population can be uniquely identified."25Toward this end, it recommends "[r]esearch into the identification and validation of more and better marker systems for forensic analysis . . . with an eye to making each profile unique."26Furthermore, the report describes some ways to show that a genotype is probably unique.27
Finally, NRC II reviews some of the social science literature pertinent to the understanding of DNA evidence by jurors.28It outlines ways that the evidence might be presented in court.29It suggests that the existing body of behavioral research sheds inadequate light on some important questions and consequently recommends additional behavioral research "to identify any conditions that might cause a trier of fact to misinterpret evidence on DNA profiling and to assess how well the various ways of presenting expert testimony on DNA can reduce such misunderstandings."30
It is too soon to be certain whether NRC II will achieve its objective of contributing "to better-informed judgments by courts and legislatures."31The very early returns are mixed.32For example, in State v. Johnson,33the supreme court of Arizona correctly relied on NRC II in concluding that the ceiling method had been generally accepted as a means of producing a generous upper bound on the population frequency of a DNA profile.34Yet, the opinion asserts that NRC I "makes [it] clear [that] the assumption of linkage equilibrium . . . is well grounded and has been proved accurate . . . ."35Evidently, the court did not understand the explanations of linkage equilibrium given in both NRC I and NRC II,36or even why NRC I floated the ceiling method in the first place.37
Although it hoped to advance the understanding of the scientific and legal issues the issues involving population genetics and statistics, the second committee knew that its recommendations were neither panaceas nor answers to all questions about the production and use of DNA evidence. Nor did the committee expect its conclusions and explanations to be accepted without question or criticism.
The essays in this symposium fulfill this expectation. Their titles speak of "accepting lower standards," of "errors and misunderstandings," and of continued "skirmishing" in a lingering war.38Many of the criticisms reflect simple (or complex) disagreements over the conclusions reached by the second committee. Others are a reaction to the limited scope of the committee's recommendations.39
Professor William C. Thompson gives a defense attorney's reaction to NRC II.40He expresses regret that the report did not demand external regulation of forensic laboratories, "blind" or "objective" reading of autoradiograms and dot blots, and retesting at separate laboratories in every case and at government expense. He also discusses the questions surveyed in NRC II as to how various probabilities or statistics associated with the laboratory work might be presented to juries.
Professor Jonathan J. Koehler focuses on one of the latter issues--using information such as proficiency test results to estimate and present to a jury the probability of a false finding of a match in a specific case.41Professor Koehler would go further than Professor Thompson in this respect. Both favor combining (a) the probability that the laboratory has declared a match when none exists with (b) the probability that a match exists but is a mere coincidence. Professor Thompson seems to agree with the suggestion in NRC II that it is acceptable to inform the jury of the latter probability, but Professor Koehler opposes informing the jury of the components of the composite probability.
Professor Richard Lempert canvasses many issues. Although Professor Lempert does not depict NRC II's calls for quality assurance and quality control in forensic laboratories and for sample splitting to permit independent retesting as seriously undermining the recommendations in NRC I,42he does detect "a failure of common sense" in NRC II's "refus[al] even to recommend that laboratory error rates, as established in blind proficiency testing, be admitted at trials."43
Professor Lempert also questions NRC II's treatment of the chance that a close relative of a defendant would have the incriminating genotype. The issue is not the scientific validity of the formulae given in NRC II for computing such probabilities. It is, once again, what statistics should be placed before the jury. Professor Lempert urges that "the defendant should be allowed to name any close relatives whom he thinks might have committed the crime" and that the state should "replace its random match statistic with a statistic showing the likelihood that at least one named relative had DNA like the defendant's unless the state excluded each named relative . . . ."44
Finally, Professors David Balding and Newton Morton concentrate on the questions of population genetics and statistics that impelled the formation of the second committee and consumed much of its efforts. Both express satisfaction with the demise of the ceiling method, but they agree on little else. Emphasizing the importance of "between-person correlations," Professor Balding questions the ubiquitous approach of using the frequency of a genotype in the general population to indicate the "evidential weight" of a DNA match.45Similarly, Professor Balding argues that NRC II's analysis of the database search issue is misguided in that it addresses the wrong question.46Finally, he dismisses the practice (supported by Professors Thompson, Koehler, and Lempert) of reporting "a match probability which adds error rates to profile frequencies" as "clearly . . . unacceptable."47
Professor Morton maintains that the debate over DNA evidence is in its final phases--an "endgame" that is "restricted to a few points."48He disputes the usefulness of the Bayesian perspective of the other commentators,49summarizes the analysis of match probabilities from the standpoint of population genetics, and examines the database-search problem. As to the last issue, Professor Morton defends the statistical reasoning in NRC II but favors the alternative suggestion in NRC I of using, whenever feasible, many confirmatory markers to establish the origin of the DNA sample.50Finally, he makes other suggestions to improve the rigor of DNA testing and confidence in it.
When the first NAS committee was formed, improving laboratory standards and performance seemed to be the issue. Before that committee could conclude its work, however, debate over population genetics overshadowed this concern, at least in the courtroom. There was little or no criticism of NRC I's discussions of the need for high standards and proficiency testing in forensic laboratories. Instead, NRC I was criticized, perhaps unfairly, for its failure to analyze the impact of population structure on the probabilities being used in court and for its resort to an unsophisticated compromise.
The discussions of NRC II collected here suggest a roughly parallel history. When the NAS decided to empanel a second committee, the hottest question was whether the ceiling method endorsed in NRC I was a satisfactory and necessary response to the fierce debate over simple product rule estimates. Following NRC II, few voices have been heard defending the ceiling method. Instead, criticisms of NRC II focus on issues that were less prominent (but perhaps of more fundamental significance) when the committee began its work.51
Whether we are seeing a strategic retreat, a light skirmish, an endgame, or the beginning of new campaign, some scientific issues involving DNA evidence--and many matters of legal policy--remain reasonably debatable. The authors of the essays presented here are outstanding scientists, statisticians, or legal scholars. They are also prolific commentators on DNA evidence.52Coming to the same questions from different backgrounds, they emphasize different points, and they do not all reach the same conclusions. One lesson to be drawn is that NRC II cannot be assumed to be correct merely because it is a consensus report of a respected organization. And, just as the source of an opinion is not a guarantee of its truth, neither is the certitude with which it is expressed. In airing a range of views on the success and failures of NRC II, Jurimetrics shares the aspiration of the authors of that report--to contribute to a wider and deeper understanding of DNA evidence.
Notes
* D.H. Kaye is Regents' Professor, Arizona State University College of Law. He served on the National Research Council Committee on Forensic DNA Science: An Update. He thanks David Balding, James Crow, Jay Koehler, Richard Lempert, Newton Morton, Stephen Stigler, and William Thompson for their helpful comments on a draft of this introduction. [BACK]
1. William C. Thompson, Evaluating the Admissibility of New Genetic Identification Tests: Lessons from the "DNA War," 84 J. Crim. L. & Criminology 22 (1993). [BACK]
2. D.H. Kaye, The Forensic Debut of the NRC's DNA Report: Population Structure, Ceiling Frequencies and the Need for Numbers, 34 Jurimetrics J. 369, 370 (1994). [BACK]
3. The NAS describes itself as "a private, non-profit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research . . . ." National Research Council Committee on DNA Technology in Forensic Science, DNA Technology in Forensic Science vi (1992) [hereinafter cited as NRC I]. Under a 1863 charter from Congress, the Academy must advise the government on scientific and technical matters. Id. In 1916, the Academy organized the NRC as its principal operating agency in providing services to the government, the public, and the scientific and engineering communities. Id.See also Kaye, supra note 2, at 370 n.2. [BACK]
4. NRC I, supra note 3, at 173-76. More precisely, it included: a professor of medical genetics; a professor of medicine, biochemistry and cell biology; two directors of forensic science laboratories; a chemical engineer; a professor of epidemiology and genetics; a professor of biology; a professor of law and sociology; a molecular geneticist; a J.D.-M.D. ethicist; a professor of forensic sciences and biomedical sciences; and a U.S. district court judge. [BACK]
5. The NAS acted at the behest of the FBI and other government agencies. NRC I, supra note 3. For one review of NRC I, see Kenneth R. Kreiling, Review-Comment, DNA Technology in Forensic Science, 33 Jurimetrics J. 449 (1993). [BACK]
6.See, e.g., Richard Lempert, DNA, Science and the Law: Two Cheers for the Ceiling Principle, 34 Jurimetrics J. 41, 42 (1993) ("No portion of the NRC report has been as controversial or as criticized as . . . the 'ceiling principle'"); Kaye, supra note 2, at 375 n.27. [BACK]
7.See, e.g., id. at 375; Eric S. Lander, Letter, DNA Fingerprinting: The NRC Report, 260 Science 1221 (1993); Lempert, supra note 6. [BACK]
8.See, e.g., Letter from William S. Sessions, director, FBI, to Frank Press, president, NAS, Apr. 16, 1993:
Since the release of the report, there have been 30 appellate decisions and in 11 of these, the decision relied on the NRC report as a basis for ruling DNA evidence not properly admissible in criminal proceedings. In addition, courts in Canada, Australia, and the United Kingdom began hearing challenges to DNA evidence--citing the NRC report--immediately following its release. This change has occurred despite the NRC report's clear recommendation that DNA evidence continue to be used.
The suggestion that the courts were rejecting evidence because of NRC I was overstated. Many of the courts would have reached the same results had there been no report in 1992. Moreover, few of them held DNA inadmissible per se. Rather, most appellate courts remanded with opinions that did not foreclose presenting the evidence of a DNA match along with a ceiling calculation, or they held that admission of the evidence was harmless error. [BACK]
9.See National Research Council Committee on DNA Forensic Science: An Update, The Evaluation of Forensic DNA Evidence v-vi (1996) [hereinafter cited as NRC II]; Letter, supra note 8 ("I ask the NRC to act quickly to resolve the controversy created over its report."). [BACK]
10. A planning group heard presentations from the FBI, the National Institute of Justice, and the Department of Energy in June 1993. NRC Planning Group, DNA Forensics: An Update, June 16, 1993 (unpublished memorandum). [BACK]
11. Sponsors were the Department of Energy, the National Institutes of Health, the National Institute of Justice, the National Science Foundation, and the State Justice Institute. NRC II, supra 9, at copyright notice page. [BACK]
13.Id. at 1. See also NRC Planning Group, supra note 10 ("The scope of the study should be narrowed. . . . The emphasis should be on statistical and population genetics issues."). Thus, the committee chair and two additional members were population geneticists; another two members were statisticians. The first committee had no members whose research careers and education were devoted to these fields. [BACK]
14. The NRC Planning Group had recommended that "the report should seek to resolve scientific, not legal issues." NRC Planning Group, supra note 10. [BACK]
15. NRC II, supra note 9, at 167. NRC II described the possible legal responses to its conclusions, but its conclusions were largely confined to describing what contemporary science could contribute to identifying the perpetrators of crimes through the analysis of DNA samples. [BACK]
19. NRC II, supra note 9, at 4 (recommendations 3.1-3.3). The first NRC committee was empaneled early in the life cycle of forensic DNA laboratories; as such, it was especially concerned with improving the performance and procedures of these laboratories, and its recommendations in this regard are more detailed and remain important. As NRC II cautions, "[s]ince the committee has not attempted to review all the statements and recommendations in the 1992 report, the lack of discussion of any statement should not be interpreted as either endorsing or rejecting that statement." Id. at 2. [BACK]
22. A more elegant approach might have been to start with the most general formulation and derive the simplifications suitable for special cases from it. See Bruce S. Weir, Invited Editorial, The Second National Research Council Report on Forensic DNA Evidence, 59 Am. J. Hum. Genetics 497 (1996). [BACK]
23. Actually, NRC I describes two types of ceilings--an "interim" ceiling formula, and a later one to be used after sampling many "[g]enetically homogenous populations from various regions of the world." NRC I, supra note 3, at 84. NRC II concludes that the ceiling methods are not needed and that they are an extravagant way to handle the likely extent of population structure. NRC II, supra note 9, at 158, 162. Ironically, much of the empirical support for this conclusion comes from data assembled by the FBI in response to the pressure of NRC I. Id. at 151. [BACK]
24. NRC II, supra note 9, at 133-35, 161. NRC I proposed computing the random-match probability only with respect to additional probes not used in the database search. NRC I, supra note 3, at 124. Although this procedure throws away information, NRC II does not suggest that it is never useful. NRC II, supra note 9, at 134 (explaining that "[t]here are different ways to take the search process into account."). It offers the Bonferroni inequality as an alternative to that approach where the database is relatively small. [BACK]
32.See D.H. Kaye, DNA Identification in Criminal Cases: Lingering and Emerging Evidentiary Issues, in Proceedings of the Seventh International Symposium on Human Identification 12 (1997). [BACK]
36. "Linkage equilibrium" is a condition that helps support the conclusion that the expected frequency of a DNA profile consisting of characteristics at several "loci" (locations on chromosomes) is the product of the frequencies of the characteristics at each locus. Another condition, "Hardy-Weinberg equilibrium," implies that the expected frequency of the two characteristics (one inherited from each parent) at each locus is twice the product of the frequency of each distinct characteristic. (If the characteristics are not distinct, that is, if the individual has inherited the same characteristic from both parents, the expected frequency is the square of the frequency of the single characteristic.)
The characteristics at each locus are called "alleles." The pair of alleles on a chromosome is a "genotype" for that locus. A set of single-locus genotypes constitutes the DNA "profile" or genotype. Thus, Hardy-Weinberg equilibrium relates to the computation of single-locus genotype frequencies, and linkage equilibrium relates to the computation of multi-locus genotype frequencies. The formula that applies when both types of equilibrium exist can be called the simple product rule. It gives the multi-locus genotype frequency as a product of the allele frequencies (and factors of 2). [BACK]
37. NRC I offered the ceiling method as a pragmatic way to moot the "considerable debate" over the effect of population substructure on both linkage equilibrium and Hardy-Weinberg equilibrium. NRC I, supra note 3, at 79. A population has substructure when it is composed of subgroups that have (1) different allele frequencies and (2) preferential mating within the subgroups. If the substructure is severe, there can be substantial departures from the genotype frequencies expected when Hardy-Weinberg and linkage equilibrium hold for the population as a whole.
Although better-informed judicial opinions might indicate some success of NRC II, there always will be courts that do not make use of the resources available to them. A scant eight months after citing and discussing NRC II in Johnson, the same court denied that it had even seen NRC II when it held ceiling estimates admissible in Johnson. State v. Hummert, 933 P.2d 1187, 1192 (Ariz. 1997). Even more puzzling, both opinions were written by the same justice. [BACK]
38. That more of the papers are negative than positive should not be assumed to reflect scientific or legal opinion generally. Almost all the papers were selected by a process that was influenced by what statisticians would call "non-response bias" on the part of those individuals who were invited to contribute and by "selection bias" in the choice of the topics to receive the most attention. Bad news still dominates the nightly broadcasts, and criticism rather than dull affirmation dominates published scholarship. [BACK]
39.See supra note 13. Some commentators evidently wanted the committee to do more than review the existing literature and data to identify the kinds of scientific information that experts can supply; they wanted a report that would have placed limitations on the use of scientifically valid evidence to avoid that evidence being systematically misused. [BACK]
40. William C. Thompson, Accepting Lower Standards: The National Research Council's Second Report on Forensic DNA Evidence, 37 Jurimetrics J. 405 (1997). [BACK]
41. Jonathan J. Koehler, Why DNA Likelihood Ratios Should Account for Error (Even when a National Research Council Report Says They Should Not), 37 Jurimetrics J. 425 (1997). [BACK]
42.See Richard Lempert, After the DNA Wars: Skirmishing with NRC II, 37 Jurimetrics J. 439, 454 (1997). Professor Thompson argues that NRC II's recommendation to offer the defendant the opportunity for retesting was "unfair and inappropriate" if not unconstitutional. Thompson, supra note 40, at 416. Professor Lempert, although not presented with Professor Thompson's arguments, reaches the opposite conclusion. Lempert, supra, at 454 (suggesting that the recommendation is "sound and promises to reduce the likelihood of laboratory error . . . substantially . . . ."). On the other hand, Professor Lempert echoes Professor Thompson's call for procedures to prevent a DNA examiner's expectations from influencing the outcome of a laboratory test. Id. at 465. [BACK]
43.Id. at 451. Although NRC II does not accept the proposition that the probability of laboratory error can be estimated satisfactorily for individual cases, neither does it expressly oppose the presentation of related figures. Indeed, recommendation 3.2 states that the results of regular proficiency tests "should be available for court proceedings." NRC II, supra note 9, at 88. As Professor Lempert recognizes, the second committee sought to confine its recommendations to matters of science, not legal policy and procedure. See id. at 41-42, 204; supra note 13. For example, it could be said that the committee "refused even to recommend that" random match probabilities computed in accordance with its recommendations "be admitted at trials." Professor Lempert argues that the second committee should have been more ambitious and made such recommendations, since it was dealing with forensic science. Although he agrees with Professors Thompson and Koehler that the laboratory's chance of having erred can be estimated satisfactorily, whether he shares Professor Koehler's view that juries should not be informed about the chance that a randomly selected person would have the incriminating genotype is not as clear. See also Lempert, supra note 6, at 54 (The false positive rate is arguably all a jury should be informed of . . . [y]et presenting the random match probability has some virtues.") [BACK]
44. Lempert, supra note 42, at 461. But cf. Lempert, supra note 6, at 55 ("The defendant . . . should be required to identify relatives who belong to the suspect population. The prosecution then could present the jury with the probability that a random man might have left the evidence DNA and the probability that one of the defendant's identified relatives might have left it. If the latter probability were high, the state could explain why particular relatives were not considered likely suspects."). [BACK]
45.See David J. Balding, Errors and Misunderstandings in the Second NRC Report, 37 Jurimetrics J. 469, 474 (1997) ("Confusingly, the report inappropriately identifies profile frequencies with match probabilities: the two are the same only if between-person correlations are neglected.") For a previous statement of this concern, see, e.g., David J. Balding et al., Comment: Some Causes for Concern About DNA Profiles, 9 Stat. Sci. 248 (1994) ("positive correlations," mostly resulting from population structure and relatives, in conditional probability that form the likelihood ratio need to be considered). Professor Balding's discussion should be read along with Professor Lempert's remarks on the treatment of relatives. [BACK]
46.Cf. Kaye, supra note 32 (noting such criticisms). Professor Lempert makes a related point. See Lempert, supra note 42, at 461. [BACK]
47. Balding, supra, note 45, at 476. He suggests, however, that "[e]rror rates observed in blind trials may well be helpful to jurors." Id. at 476 n.21. [BACK]
48. N.E. Morton, The Forensic DNA Endgame, 37 Jurimetrics J. 477, 478. [BACK]
49. Consequently, he dismisses efforts to blend the probability of police and laboratory error or fraud with the probability of coincidentally matching genotypes on the ground that the former probability "cannot be objectively included in the likelihood ratio." Id. at 481. [BACK]
51. There was considerable academic writing about the importance of laboratory error and close relatives, and on Bayesian methods of combining and presenting information. Indeed, much of this writing came from the contributors to this symposium. Still, it is fair to say that these issues were not receiving the same attention in scientific journals and in courtrooms as was the issue of population structure. [BACK]
52. Most, but not all, have served as expert witnesses for defendants. [BACK]
