Experts and Errors (1)

Question 1

What is a misscarriage of justice?

Answer 1

= wrongfull conviction
(Saks & Koehler, 2005)
- aquitting someone athough they are guilty is an error of impunity

Question 2

LG: What are jurors believes regarding reliability of forensic evidence?
Schweitzer & Nuñez, 2018
Study I (what is actually used)

participants
method
results

Answer 2

PARTICIPANTS: legal professionals from the US
METHOD: Survey
- PP were given a list of
66 pieces of evidence generated by psycho-legal researchers
→ What of the evidence was commonly used at homicide trials in general?
→ What was used in there last trial? (to exclude availability/representativeness heuristic)
FINDINGS: 10 pieces of evidence were
selected as being common by 50+ % of the PPs
→ crime-scene pictures, witness to the crime, diagrams of injuries, police officer expert testimony, forensic expert testimony, weapon, fingerprints, maps, audio confession, and video confession

Question 3

LG: What are jurors believes regarding reliability of forensic evidence?
Schweitzer & Nuñez, 2018
Study II

Answer 3

PARTICIPANTS: lay people (could-be jurors)
METHOD: list with 41 pieces of evidence was given
→ PP aks: Which piece of evidence was most important to them to determine if someone was guilty or not?

FINDINGS: Top 10 most important pieces of evidence:
1st - DNA
→ has received scientific attention, but fallibility of the method is under-estimated (most persuasive)

• followed by fingerprints, the
  weapon, video records, crime-scene photos, gunshot residue, bodily secretions, video confession,
  testimony from a forensic expert, and
  eyewitness testimony

Question 4

LG: What are jurors believes regarding reliability of forensic evidence?
Schweitzer & Nuñez, 2018
Study III

Answer 4

AIM: replicate findings of study 2 using a different method –> whether using
just the 10 most important pieces of evidence alters mock jurors’ perceptions of importance
METHOD:
- first did a pilot study shorten the list they could choose from
- Process tracing method:
mock jurors
chose which folder of evidence they wanted
to see first in line with percieved importance –> prioritize!

FINDINGS:
DNA was viewed first most often, but eyewitness, video confession and
fingerprint analysis did not significantly differ from each other
→ a decision tree showed that when a person looked at DNA evidence first, this was mostly followed by looking at video confession, then fingerprints and then eyewitness

• Limitation: very small amount of pieces of evidence

Question 5

LG: What are jurors believes regarding reliability of forensic evidence?
Schweitzer & Nuñez, 2018
General

• Findings
• Limitations

Answer 5

• DNA = top notch in jurors eyes
  • the other evidence pieces have less of a consistent importance
  ➔although inconsistent fingerprints, eyewitness testimony and video confession evidence are also important to mock jurors

→ might have looked at the evidence as a whole
→ previous knowledge and UNDERSTANDING of PP may have influenced their decision making
→ limited to evidence from homicide trials
→ only applicable to jurors (US) not to professional judges (elsewhere)

Question 6

LG: What are jurors believes regarding reliability of forensic evidence?
Schweitzer & Nuñez, 2018
General
- jurors weighing of evidence (existing lit.)

Answer 6

Eyewitness testimony
- most convincing but also most unreliable piece of evidence →reliability is overestimated (when witness very confident)

Expert testimony
- mixed findings: some jurors under and other over-estimate scientific expert testimony
→ broad range of the type of experts make it hard to draw conclusions on the degree to which jurors weight expert testimony
→ prevalence of expert testimony has increased over time but varies across locations

Visual evidence
- influences judgment → if visual evidence is given, the defendant is found more guilty than when there is no evidence

Question 7

LG: What are jurors believes regarding reliability of forensic evidence? 
Ribeior et al., 2019
- Whats is the CSI effect?
- What are the presumed biases? 
- Any evidence?

Answer 7

= people who view crime showes on TV have a over-exegerated view on how forensic science works

→ CSI effect is thought to influence jurors’ verdicts by either:
a) Pro-defense bias:
“demanding”/ expecting there to be forensic evidence = higher rates of acquittal if not
b) Pro-prosecution bias:
unrealistically high faith in accuracy of forensic
science = higher rate of conviction if present

• previously: mixed result
• Ribeiro et al. found no evidence for the CSI effect

Question 8

LG: What are jurors believes regarding reliability of forensic evidence?
Ribeior et al., 2019
- Potential problems with juror beliefs
Story model of jury decision-making
Base-rate neglect

Answer 8

• jurors are lay people with no understanding of science especially forensic science in particular → understanding their believes is crucial

             > jurors may incorporate evidence presented with their pre-existing knowledge to form a narrative representation of the evidence (similar to the judge in the Amanda Knox)  --> different jurors may come to different conclusions 

→ Base-rate neglect:
people ignore relevant base-rate information in favour of more specific but irrelevant information
• representativeness heuristic:
make intuitive judgments of probability based on how similar to a prototype
• for accurate estimate of probability need to override intuitive judgment –> not always the case when estimating the accuracy of forensic technique

–> previous studies have shown an over-estimation of the reliability and validity of different forensic techniques

Question 9

LG: What are jurors believes regarding reliability of forensic evidence?
Ribeior et al., 2019
- forensic evidence (overview)

Answer 9

• DNA Gold = standard of forensic techniques: small random match probabilities, errors are not too likely but still possible (human error and biases) → no proper validation experiments
• Fingerprint experts perform way better than lay persons, however studies revealed that examiners can disagree about the number of identifying features in fingerprints → susceptible to contextual bias
• Bite marks: most controversial of the forensic techniques → reported to not live up to the standards of scientific validity and reliability

Question 10

LG: What are jurors believes regarding reliability of forensic evidence? –> CSI Effect
Ribeior et al., 2019
- Study

Aim
Hypothesis
Method
Results

Answer 10

AIM: investigate beliefs about human involvement/ error rates in forensic science

HYPOTHESES:
1st & 3rd - underestimation of human involvement/ error rate (general and specific techniqhes of FS)
2nd & 4th - crime show viewing and estimation of error rate neg. correlated & positive with accuracy (CSI effect)
5th - people ignore base-rate and rely on representstive heuristic

METHOD:

• pp = Aussies
• imangine you are at a crime scene with for. ev.!
• questions about accuracy, involvement and crime show viewing habits (questionaire)

FINDINGS:

• PP believed that there was a substantial likelihood of error and human involvement at each stage of the forensic science process (rejected)
• PP estimates of accuracy for individual techniques was inconsistent with the base-rate likelihood of an error occurring at each stage of the process they had just provide (accepted)
• PP either underestimated or overestimated the accuracy of foresic techniques –> still no good understanding
• CSI effect mostly rejected

Question 11

What is a cognitive bias?

Answer 11

= systematic errors in thinking when processing and analyzing information that affects subsequent decisions and judgment
–> e.g.: hindsight bias

Question 12

What is a heuristic?

Answer 12

= mental shortcuts that lead to reasoning error

Question 13

What is a fallacy?

Answer 13

= invalid conclusions that stem from flawed reasoning

Question 14

What is the Dual process theory?

Answer 14

siehe Croskerry fig. 7

Question 15

LG: What are sources of miscarriages of justice?

- How often is this source forensic science?

Answer 15

• Saks and Koehler analysed the Innocence Projects exoneration cases
• -> forensic science testing error are the second most involved causes for miscarriages of justice (63%) only toped by eyewitness errors

Question 16

LG: What is the origin of errors of forensic experts?

• practitioner error
• instrument (technological) error
• statistical error
• Method (technique) error

Answer 16

• Mistake or operator (human) error: it may be random or systematic and mostly unintentional
• -> hard to reduce
• -> not an error in the scientific sence
• Difference btw an indicated instrument value and the actual (true) value: instruments are calibrated against a standard but often deviate a little
  → Can be measured statistically and can be minimized by proper maintenance and
  calibration of instruments Difference btw computed/measured and true value:
  estimated by standard error
• Difference btw. a predicted and a obtained value (scientific definition of error vs. mistake)
• Limitations of a method:
  How well does the method measure the item? (always limitations)
  –> often overlap among different items
  → no errors per se

Question 17

LG: What is the origin of errors of forensic experts?
Misunderstanding error
- Claiming a zero-error rate
- Claiming that an error rate cannot be estimated
- Attempting to calculate error rates post facto from
activities that were not intended for that

Answer 17

• claim is unscientific, because there is always an error rate
• not true and results of a lack of proper testing, insufficient statistical training or misunderstanding of the meaning of error
  → fear that by acknowledging error rates, the power of the method will be diminished

Not acceptable, because results are taken out of context and used for something completely unrelated
→ results of random studies not intended to test error rates, often do not show the methods accuracy or reliability

Question 18

LG: What are the reliability and validity of different forensic methods?
- traditional way

Answer 18

→ many methods didn’t meet scientific standard => no safeguard against biases => miscarriage of justice

2 major principles in classical forensic identification science:
1. principle of uniqueness:
nature never repeats itself - impossible to prove
2. principle of individualization:
every trace can be related to a unique source
–> when two items have many characteristics in common that is can’t be due to chance –> the same

     a) class characteristics: 
general characteristics that of one group (can exclude people who are not from the group but cannot determine who it is)

 b) individual characteristics:  exceptional characteristics that establish uniqueness 

–> assumptions are problematic:
• to be sure that two items are the same need to refer to many alternatives
• definition is circular: we cannot establish uniqueness (we have to know that all swans are white to know that a black swan is unique)

(Linked to forensic science:

“Leap of faith”
a lot corresponding pattern assume it a match = jumping to conclusions
—> however, we cannot individualize –> we never know how unique a corresponding pattern actually is)

Question 19

LG: What are the reliability and validity of different forensic methods?
- new way: DNA as a model

Answer 19

DNA typing can serve as the model for traditional forensic science in 3 ways:

1. DNA typing technology was an application of knowledge derived from core scientific disciplines
2. Courts and scientists scrutinized applications of the technology in individual cases –unscientific practices were rooted out
3. DNA typing offered data-based, probabilistic assessments of the meaning of evidentiary matches Strength of DNA: uses a statistical approach-based on population genetics theory and empirical testing

→Experts evaluate matches btw suspects and crime scene DNA in terms of the probability of random matches across different reference populations –> Traditional forensic sciences should also use this approach
→Fingerprinting could be one of the first areas to make the transition to this approach since there already exist a database (other areas should probably not be used at all in court because of their bad false positive error-rate)
→The scientific foundation would be strengthened by estimating error rates: best way to do so is to conduct blind, external proficiency tests using realistic samples (external proficiency test: make judgment about a sample whose properties are known)

Question 20

LG: What are the reliability and validity of different forensic methods?
- new way: Daubert criteria

Answer 20

= criteria for scientific evidence to meet if it is to be submitted to the jury

1. underlying theory, technique, methodology or reasoning is tested and should be testable
2. the theory was peer reviewed and published
3. the potential error rate of the technique should be known
4. there should be maintained standards controlling the technique’s operation
5. the method or technique should pass the general acceptance test of the scientific community

Question 21

LG: What are the reliability and validity of different forensic methods?

• the paradigm shift (summary)

Answer 21

–> DNA analysis brought a paradigm shift into forensic science

• before: principles of uniqueness and individuality
  now: Daubert criteria
• before: categorical approach
  now: probabilistic
• before: jumping to conclusion “leap of faith”
  now: elimination (á la Hercules Perot)

→resolved old cold cases + categorically eliminate
(=ideologically neutral, because it can provide incriminating and exonerating)
→ acquired firm theoretical basis and thereby differs from all other types of forensic identification evidence
→ increased our understanding of potential and actual dangers of relying too heavily on the use of eyewitness testimony + types of forensic or technical expertise in judicial fact finding

=> with DNA analysis the field of forensic science became more scientific and less prone to error, hence less prone to miscarriages of justice