STRmix comments on the PCAST Report to the President Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods, September, 2016

We support the finding of the President's Council of Advisors on Science and Technology (PCAST, hereafter “the Council”) on CPI based methods, and are pleased to see that the Council is encouraging the forensic science community to move to more appropriate methods based on probabilistic genotyping for the DNA analysis of complex-mixture samples.  

The Council states that probabilistic genotyping software programs “clearly represent a major improvement over purely subjective interpretation”.  We agree with this finding.  The developers of STRmix™ are leading researchers in this area and their research is extensive.  Specifically, all significant portions of the statistical algorithms and underlying scientific principles behind STRmix™ have been published in peer reviewed scientific literature [1]. 

The Council also states that appropriate evaluation of proposed methods should consist of studies by multiple groups, not associated with the software developers.  Whilst we assert that the scientific research carried out by the STRmix™ developers has the utmost scientific integrity, we also fully support independent validation. 

We wish to point out that internal validation studies have been undertaken by every forensic biology laboratory that is now using STRmix™ in casework.  At the time of writing, these now number twelve laboratories in the United States, six in Australia, and one each in Canada, England, Scotland, Ireland and New Zealand.   Internal validation studies are required for a laboratory’s accreditation (such as ASCLD/LAB, ANAB, UKAS or NATA), and are therefore scientifically scrutinised and robust.  In many instances internal validation has been undertaken more than once within a laboratory for different multiplexes or different versions of STRmix™, mostly specifically designed to meet or exceed the SWGDAM guidelines for the validation of probabilistic genotyping software [2].  In addition to the 23 live laboratories, there are many more laboratories that are part way through internal validation projects.

The Council may be unaware of these internal validation studies because to date they have not been published. A collated study that compares and contrasts internal STRmix™ validations by multiple laboratories is in preparation and we hope that this will be accepted for publication in the future.   Publication of internal validation has been difficult to achieve in the past in our experience. 

STRmix™ encourages third party evaluation of the software and would gladly cooperate with researchers carrying out a verifiably independent comparative study on the performance of the different models of DNA profile interpretation. 

We know of examples of work, carried out independently of the developers, presented at recent international conferences:

ANZFSS 2016 Symposium, Auckland, New Zealand:

• Familial searching of DNA profiles from skeletal remains cases using STRmix in the northern territory - Kate Cheong-Wing, Northern Territories
• Determining the expected variation of likelihood ratios resulting from the analysis of a range of single source and mixed DNA profiles using the probabilistic genotyping STRmix™ software - Todd Bille, United States ATFE
• A direct comparison of the STRmix™ and TrueAllele® expert forensic systems - Zane Kerr, Forensic & Analytical Science Service, NSW

ISHI 27 2016, Minneapolis, USA:

• Johnny’s Got a Gun, or Is It Tom, or Maybe Bill: Combating Complex DNA Mixtures Obtained from Gun Swabs Utilizing the Probabilistic Genotyping Software STRMIX™ - Rachel Oefelein, DNA Labs International
• The Lone Star Mix: Identifying and Correcting DNA Mixture Cases as a Matter of Shared Ethical Duty - Lynn Robitaille Garcia, Texas Forensic Science Commission

AAFS 2016, Las Vegas, USA:

• Multi-Software Interpretation of Complex Mixture DNA Profiles: A Comprehensive Approach to Explaining DNA Interpretation Results in Courtrooms - Eugenio Alladio, Paolo Garofano, Italian Academy of Forensic Science

ISHI 26 2015, Grapevine, Texas, USA:

• Two Years Later: A Reflection on the Implementation of STRmix™ in a High-Throughput DNA Laboratory – Zane Kerr, Forensic & Analytical Science Service, NSW
• The Dangers of Not Assuming Contributors – Why the Goal of “Conservative” in Forensic DNA Statistics Should be Dropped in Favor Of Being “Informative” – Joel Sutton, Timothy Kalafut, Lindsey Smith, Curt Schuerman, and David Diekema, United States Army Criminal Investigation Laboratory

ANZFSS 2014 Symposium, Adelaide, Australia:

• Case Study: From blood groups to STR Mix - How improved DNA profiling technology provided crucial evidence in a cold case homicide – David Bruce, Forensic & Analytical Science Service, NSW

The Council concluded that “published evidence supports the foundational validity of analysis, with some programs, of DNA mixtures of 3 individuals in which the minor contributor constitutes at least 20 percent of the intact DNA in the mixture and in which the DNA amount exceeds the minimum required level for the method.”  We question the scientific foundation for this limit given the information already available in the literature [1, 3-6].

In conclusion, we are encouraged to see that the Council has recommended moving to more appropriate methods of DNA profile interpretation based on probabilistic genotyping.  We hope that a fuller survey of the validation data available for software such as STRmix™ will quickly widen the range of “foundational validity” in the opinion of the Council.

References

[1] Bright J-A, Taylor D, McGovern CE, Cooper S, Russell L, Abarno D, et al. Developmental validation of STRmix™, expert software for the interpretation of forensic DNA profiles. Forensic Science International: Genetics. 2016;23:226-39.

[2] Scientific Working Group on DNA Analysis Methods (SWGDAM). Guidelines for the Validation of Probabilistic Genotyping Systems. 2015. [Accessed] 3 October 2016. http://media.wix.com/ugd/4344b0_22776006b67c4a32a5ffc04fe3b56515.pdf(external link) 

[3] Taylor D. Using continuous DNA interpretation methods to revisit likelihood ratio behaviour. Forensic Science International: Genetics. 2014;11:144-53.

[4] Bright J-A, Taylor D, Curran J, Buckleton J. Searching mixed DNA profiles directly against profile databases. Forensic Science International: Genetics. 2014;9:102-10.

[5] Bright J-A, Curran JM, Buckleton JS. The effect of the uncertainty in the number of contributors to mixed DNA profiles on profile interpretation. Forensic Science International: Genetics. 2014;12:208-14.

[6] Taylor D, Buckleton J. Do low template DNA profiles have useful quantitative data? Forensic Science International: Genetics. 2015;16:13-6.