Should Medical Experts Giving Evidence in Criminal Trials Adhere to EFNSI Forensic Guidelines in Evaluative Reporting

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The paper is well written and represents a valuable contribution.  I have only a few comments:

Line 142 ""tat" -> that"

Line 176 spell out Andrey Kolgomorov

Line 219: intoxicated is used twice when assaulted was intended.

Line 259-263: this sentence is difficult to understand.  Perhaps a period after guilt would help.

 

 

Author Response

Reply to Reviewer 1.

 

Thank you for your helpful comments.

Line 142 ""tat" -> that"

This typo has been corrected

 

Line 176 spell out Andrey Kolgomorov

I have spelt out Andrey Kolgomorov, as suggested.

 

Line 219: intoxicated is used twice when assaulted was intended.

I have corrected this line to

Of those who assault, half are intoxicated:           

                     3

Of those who are intoxicated, 5% commit assault:

 

Line 259-263: this sentence is difficult to understand.  Perhaps a period after guilt would help.

I have split this sentence into three sentences:

 The presumed error in the original trial was to mistake the LR for the odds of guilt.  That is “prior neglect” or “baseline neglect” described by Kahneman. Prior neglect is equivalent to setting the prior odds at 1, or believing that before the shoe evidence was considered, with no evidence of guilt or hypothesis to support guilt, the suspect has a 50% chance of being the murderer.

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

This is a relevant paper very well written and it should be accepted. In my opinion is an outstanding paper.

The example of sexual behaviours to illustrate the need for a scientic approach of the value of the evidence is not bad (I could add a few more in other forensic areas).

The only question in my opinion is that it would important to give some references on the mathematical basis of the ENFSI guidelines (i.e. the original Ian Evett contributions)

Author Response

Reply to Reviewer 2

 

Thank you for your kind review and helpful feedback.

 

I entirely agree that I should have documented Ian Evett’s contribution and the theoretical basis of the guidelines.  To this end, I have added the following paragraph (in red) to the introduction:

…….exonerated in 2023 after 21 years in jail[4].  Poor forensic standards and an asymmetric prosecution-orientated presentation of forensic evidence in court, focused on opinion rather than evidence, were blamed[5,6].  Ian Evett, who had championed the use of Bayesian Statistics in forensic science since the 1980s[7], led the development of the European Network of Forensic Science Institutes standards, requiring forensic scientists to consider all possible hypotheses formally.  Specifically, where there are two hypotheses and one variable, the ratio of the odds of posterior and prior hypotheses, the Bayes factor, is identical to the likelihood ratio, the ratio of the probability of finding the evidence under each hypothesis [7,8]. The requirement to quote these statistics fulfilled the requirement of balance and independence.  Similar guidelines were developed by the American Standards Board (ASB) of the American Academy of Forensic Science.

In 2002, Daniel Kahneman was given the Nobel Prize in Economics for work conducted over 40 years with the late Amos Tversky “for having integrated insights from psychological research into economic science, especially concerning human judgment and decision-making under uncertainty “[7]……….

7	Evett, Ian W. "Bayesian inference and forensic science: problems and perspectives." Journal of the Royal Statistical Society. Series D (The Statistician) 1987 36.2/3: 99-105.
8	Evett I. The logical foundations of forensic science: towards reliable knowledge. Philos Trans R Soc Lond B Biol Sci. 2015;370(1674):20140263. doi:10.1098/rstb.2014.0263

 

 

I did not wish to overemphasise  the fact that the guidelines were based on (the odds version) of Bayes’ theorem because there were other themes in my paper:

1. Bayes’ theorem as set out in some of the early papers was presented as a rather mathematically technical formula, [and referred to in judges’ rulings disparagingly as a formula], and therefore not comprehensible by non-mathematicians. I wanted to introduce Bayes’ theorem as an elementary matter, comprehensible by pre-sixth form school children, comprehensible by example and inspection of a Venn diagram.
2. The Likelihood ratio in the development of EFNSI guidelines came from the odds version of Baye’s theorem, but I wanted to introduce it as a definition of evidential probity, independent of Bayes’ theorem that was also the Bayes’ factor in two-variable conditional probability. First, the EFNSI guidelines could be applied and used in directions to the jury without them needing to know anything about conditional probability (though Judges and lawyers should or must know about elementary probability).  Second, the LR retains its validity as a measure of evidential probity (heuristically at least)  when items of evidence are not independent and directed acyclic graphs (DAGs)are required.  (I prefer the term DAGs to Bayes’ nets, a)because Bayes is off-putting and b) because DAGs are all about conditional probability but not about Bayes’ theorem, and it is confusing.)
3. There is a big issue about how DAGs should be handled in court. I do not agree with Norman Fenton’s view that its like trusting a calculator to do long division.  There is a whole discussion to be had about rules of evidence and whether it is appropriate to discuss the factoring of variables in court.  But that discussion can only be had when elementary probability theory is part of mandatory training for lawyers (and expert witnesses).

Author Response File: Author Response.docx