Topic 7 - Bias Flashcards
A key requirement of the Forensic Science Regulator’s Codes of Practice and Conduct for Forensic Science Providers and Practitioners (the Codes) is that they act with…
- Honesty
- Integrity
- Objectivity
- Impartiality
Categories of cognitive bias
- Experimental bias
- Confirmation bias
- Anchoring effects or focalism
Expectation bias
Where the expectation of what an individual will find affects what is actually found.
Confirmation bias
- Closely related to expectation bias
- Whereby people test hypotheses by looking for confirming evidence rather than for potentially conflicting evidence
Anchoring effects of focalism
Occur when an individual relies too heavily on an initial piece of information when making subsequent judgements, which are then interpreted on the basis of the anchor.
Causes of bias?
- Overconfidence
- Recency
- Salience
Overconfidence
Causes of bias
- Some of us are too confident about our abilities and this causes us to take greater risks.
- Experts are more prone to this bias than laypeople since they are more convinced that they are right
Receny
Causes of bias
The tendency to weigh the latest information more heavily than older data
Salience
Cause of bias
Our tendency to focus on the most easily recognisable features of the situation or concept
Availability heuristic
People who overestimate the importance of information that is available to them.
Bandwagon effect
The probability of one person adopting a belief increases based on the number of people who hold that belief.
Blind spot bias
Failing to recognise your own cognitive biases is a bias in itself
Choice supportive bias
When one chooses something which we tend to feel positive about
Clustering illusion
Tendency to see patterns in random events.
Confirmation bias
We tend to listen only to information that confirms our preconceptions.
Conservatism bias
Where people favour prior evidence over new evidence or information that has emerged
Information bias
- The tendency to seek information when it does not affect action.
- More information is not always better
Ostrich effect
The decision to ignore a dangerous or negative information by burying one’s head in the sand.
Outcome bias
Judging decisions are based on the outcome rather than how exactly the decision was made.
Selective perception
Allowing our expectations to influence how we perceive events.
Survivorship bias
An error that comes from focusing only on surviving examples causing us to misjudge a situation.
Criminal procedure rules - Part 19
An experts duty to the court includes the following:
* An expert must help the court to achieve the overriding objective – by giving an opinion that is objective, unbiased and wihtin the experts area of expertise.
* This duty overrides any obligation to the person from whom the expert receives instructions or by whom the expert is paid
The role of the forensic expert
- To evaluate scientific findings and the results of analytical tests in the context of the relevant case circumstances.
What criteria should the exprts opinion meet?
An expert opinion should meet the following criteria
* Balanced
* Robust
* Logical
* Transparent
Categories of cognitive bias
Effects
- Contexttual bias
- Role effects
- Motivational bias
- Reconstructive effects
Contextual bias
Cognitive bias
Where someone has other information aside from that being considered, which influences (either consciously or subconsciously) the outcome of the consideration.
Role effects
Cognitive bias
- Are where scientists identify themselves within adversarial judicial systems as part of either the prosecution or defence teams.
- This may introduce subconscious bias that can influence decisions, especially where some ambiguity exists.
Motivational bias
Cognitive bias
Occurs where, for example, motivational influence on decision making, results in information consistent with a favoured conclusion tending to be subject to a lower level of scrutiny than information that may support a less favoured outcome.
Reconstructive bias
Cognitive bias
- Can occur when people rely on memory rather than taking contemporaneous notes.
- In this case people tend subsequently to fill in gaps with what they believe should have happened.
- And so may be influenced by protocol requirements when recalling events some time later from memory.
Risks are bias are lower when…
Risks are bias are lower when results are clear and unambigous.
Risks of bias are greater when…
Results
Risks of bias are greater when results are:
* Complex
* Poor quality
* Increased reliance on subjective opinion
Risks of bias are lower when..
Approach
Risks are lower when there is a methodical approach with defined standards built on principles that have been tested and validated.
Risks of bias are greater when…
Approaches
When the approach is unresearched and personal to the practitioner.
Risks of bias are lower when…
Practitioners
When practitioners and checkers are:
* well trained
* Experienced
* Continuously meet acceptable standards of competence
Risks of bias are higher when…
Practitioner
When practitioners and checkers are:
* inexperienced
* Unmonitored
* Left to adopt their own approach
Risks of bias are lower when…
Checking
- When interpretation is checked by a competent peer.
- who conducts a separate interpretation fully independent and without influence from the reporting scientist.
Risks of bias are higher when…
Checking
When checking is less rigorous and or conducted collaboratively.
Debias
The reduction or elimination of the impact of bias in decision making and problem solving.
Photogrammetry
The practice of obtaining reliable information about physical objects through the processes of recording measuring and interpreting photographic images.
Psychological contamination
Exposure to other information that is irrelevant to the assessment but that introduces sub conscious bias into the findings.
Reconstructive effects
The tendency when people rely on memory to fill in gaps on recall with what they believe should have happened.
Threats to impartiality
- Being the sole reviewer of their critical findings
- Being over-familiar with or trusting another person instead of relying on objective evidence
- Having organisational and management structures that could be perceived to reward, encourage or support bias where
- A culture of performance measurement
- And time pressures could potentially pressurise examiners into biasing decisions.
Why is it important to have the whole picture?
- Important for constructing and testing relevant hypotheses and propositions.
- However, if the whole case file is handed over to an analyst with all the detail, there is the perception that bias could have occurred and may be open to challenge in court
How can suseptibility to psychological and cognitive influences be assessed?
- This may be achieved through a proficiency-testing programme
- Utilising mocked up casework samples for which the expected outcomes of testing and evaluation are known.
- Whilst blind trials are the gold standard in providing the most reliable indicator of real-life performance, in reality they can be very time-consuming.
- Good practice adopted by many laboratories is to undertake a mixed programme of both declared and undeclared trials, with the proficiency of all individuals tested on a regular basis
Awareness, training and competence assessment
- Practitioners need training in how the evaluation procedures assist in overcoming some of these risks in their respective roles, as well as what residual risk may remain.
- Given that susceptibility to psychological and cognitive influences varies between individuals, there may be merit in assessing these susceptibilities as part of the recruitment or selection procedures for new staff.
Avoiding reconstructive effects
- The taking of contemporaneous notes or technical records is another stipulation in the Codes (section 15.2.3)
- Adherence with this requirement wherever it is practicable to do so at all stages in the collection and processing of forensic evidence provides the best safeguard against potential reconstructive effects .
Avoiding role effects
- This is where scientists are subconsciously influenced by acting on behalf of the defence or prosecution.
- Difficult to demonstrably eliminate given the adversarial nature of the CJS within the UK.
- These pressures can also apply in the case of police laboratories in providing services to an internal customer.
Forensic Process outline
- Define the requirement
- Develop an examination strategy
- Agree the examination strategy with the client
- Lead scientist instructs analyst to carry out forensic examinations and analyses
- Review the quality and content of examination results
- Compare the results with the reference samples and marks.
- Evaluate and interpret the scientific findings and analytical tests
- Second expert verifies the findings
- Communicate the scientific findings and analytical tests
During the forensic outline, what responsibility does the expert have?
Reveal
It is the responsibility of the expert to:
* Reveal their work (disclose)
* Record all information received
* Record details of interpretation
When working in the evaluative mode, the expert goes through:
When working in the evaluative mode, the expert goes through:
* A formal process of pre-assessing the expected probabilities
* For a realistic range of possible outcomes
* In as many or as few categories as is sensible for the examination
* Recording their opinions
Realistc list of outcomes
- At source level, and certainly at activity level, it may be that each category in the realistic list of outcomes is considered firstly under the assumption that the prosecution hypothesis is true, and secondly under the assumption that the defence hypothesis is true.
- These are used to provide an expected outcome that may be either qualitative or quantitative with the latter expressed as a likelihood ratio.
Recommended good practise
- Identify whether the scientist’s role in the case is investigative (for example, intelligence) or evaluative (judicial)
- Seek clarity on what are the issues, the purpose and how this fits into the hierarchy of sub-source (for example, touch DNA), source, activity and offence level propositions.
Develope an evaluative examination strategy
- Formulate relevant prosecution and defence alternatives based on the case circumstances and information provided.
- Consider any agreed assumptions that are used in formulating these alternatives.
- Use assessment of possible outcomes to determine which tests are most informative and discriminating.
- Use this pre-assessment to assign a weight to an exhaustive list of possible outcomes, giving the expected outcome for each, expressed as a LR where these are quantitative.
- This approach provides clarity on the alternatives being considered, and the pre-assessment of weight for all outcomes avoids the potential bias of using the observed results to assign weight of evidence.
Carry out forensic examinations and analyses
- Review the quality and content of examination results.
- Decisions on the suitability of the results and marks for later comparison are made at this stage, to avoid post-comparison rationalisation of opinion on quality.
- Compare the results with the reference samples and marks.
- The quality and suitability of the questioned result has already been assessed so this is not influenced by the reference result.
Evaluative and Interpret scientific findings and analytical tests
- Mitigate the confirmation bias by using the LR or qualitative expectation that was assigned to each outcome before the examinations and tests were performed
- Pre-assessment enables the scientist to explain how the weight of the evidence has been assigned
- Provide details of the assumptions that have been made
- Give the basis of the expert opinion and specify the propositions considered, with the reasoning for these, based on the case context
- Include any limitation of the opinion
- Where there is a range of opinion on the matters dealt with in the report, the expert’s report must summarise the range of opinion and give reasons for the expert’s own opinion as required by the Criminal Procedure Rules
Recommended Good Practise
Second expert
- Verification by a second expert - Independent review at this stage in advance of communicating the result to the client
- Communicate the scientific findings and analytical tests
The police response to a reported crime requires many factors to be taken into consideration:
The police response to a reported crime requires many factors to be taken into consideration:
* Preserving the scene
* Securing evidence
* The speed of response
* The proportionate use of resources based on the seriousness of the crime
* All are overridden by the most pressing priority -the preservation of life
Crime scene activities and risk of bias impact
The potential impact may be significant, for example:
* It could result in failure to secure the required evidence if a crime scene.
* An investigation could be closed prematurely, resulting in crucial evidence being lost.
* It could mislead an investigation by investigators focusing too early and incorrectly on a false lead, and other evidence is potentially overlooked.
* If undertaken incorrectly activities could result in ‘psychological contamination’ of evidence downstream in the forensic analysis and interpretation processes.
Oppoertunities for cognitive bias
Gathering information prior to scene attendance.
* Prior to scene attendance information is gathered from any available source regarding the incident to be investigated.
* This may include witness or victim accounts as to what is alleged to have happened and by their nature these may.
The cognitive processess entail…
The cognitive processess entail:
* Determining locations and boundaries of the scene
* The entry/exit points of the offender
* Information received and inferences
What factors can cause bias in the cognitive process?
- The past experiences of an individual on which they may base their decisions are subjective.
- Convenience may be more relevant and have more impact in real life. - For example, establishing the boundary by taping from lamp post to lamp post is commonplace simply because they are already there.
The cognitive process
Creating a record of the scene
- Selection of equipment
- Decisions on which images to capture
- An assessment of the current case needs plus some anticipation of future needs.
- Depending on how the written record is crafted, there is a risk that contextual or confirmation bias may be introduced. downstream in the investigative process.
Undertaking forensic examinations at scenes
- Requires an understanding of the investigative needs of the case to observe, discover and recover evidence to meet both present needs and those anticipated for the future.
- If guidance for these decision-making processes is not explicitly documented then actions taken at this stage are largely reliant on the examiners intuition and tacit knowledge which in turn are susceptible to bias.
Why is communication of findings to others through written reports rather than verbal updates preferred?
- Whilst slower, is preferable as the former provides less risk of introducing bias into the transfer of information.
- Is more transparent about the decision making process.
DNA Mixtures
- Items are received along with case information and questions to be addressed by the scientific work.
- The case information is used to direct the DNA recovery and analysis strategy, ideally within a framework of appropriate propositions.
- Findings are checked by a competent colleague/peer
- A statement or report is issued
- The scientist may be called to court to give oral testimony
DNA Mixtures
Non complex DNA results
If non-complex DNA results are obtained that match a suspect, an appropriate random match probability or likelihood ratio (LR) estimate is assigned.
DNA Mixtures
Complex mixed DNA numerically
If complex mixed DNA results are obtained that can be numerically evaluated the probability of the mixed result is calculated under appropriate prosecution and defence hypotheses and a LR is assigned
DNA Mixtures
Complex DNA non-numerical
If complex DNA results are obtained that do not lend themselves to statistical evaluation, in some circumstances, a qualitative assessment is made and an opinion about the significance of the DNA results can be put forward.
R v Pickering (2012)
Background
- In 2012, Pickering was convicted at the Crown Court on one count of sexual assault on a girl under the age of 13
- He was sentenced to 30 months imprisonment
R v Pickering (2012)
The circumstances
- Pickering’s partner had an arrangement that they would babysit for each other
- On 9 September 2011 she left her children, P (a girl of 12), D (a boy of 8) and M (a girl of 8) with Pickering and his partner for them to babysit with their own child, C aged six
- Between 2230 and 2330, Pickering’s partner went to bed leaving him with the children. He had consumed a quantity of beer
R v Pickering (2012)
Victims Evidence
- ‘P’s evidence is that Pickering touched her vagina over her clothes
- Afterwards he asked to go into the kitchen with him
- He lifted up her top and touched and sucked her breasts and gave her a love bite
- He then pulled her top down and removed her lower clothing and began licking or sucking her vagina
R v Pickering (2012)
Wedgie
- She denied that Pickering had given her a wedgie and denied making the allegation up
- The following morning the victim showed her father the red mark on her neck and told her parents that Pickering had sucked her breasts and vagina
- Victims evidence was that he had tickled his son and given him a wedgie
- Pickering denied touching ‘P’ inappropriately
R v Pickering (2012)
Forensic analysis
Four swabs were taken from P
1. Left breast swab
2. Vulva swab
3. Knickers
4. Vest top
R v Pickering (2012)
The underwear
- Visible cream-coloured staining was visible in the gusset.
- Amylase was found to be present in a high concentration.
- A mixed DNA profile contributed to by at least three people
- All the components of the DNA profiles of ‘P’ and all but one of the components of the DNA profile of Pickering were present
- There were two components definitely present and three other components possibly present that could not have come from P or Pickering
What is amylase?
Amylase is a main constituents of saliva where it is usually present in high, though variable concentrations although it is present in other body fluids, a high concentration is indicative of the fluid being saliva
R v Pickering (2012)
Left breast
- Not possible to determine if saliva was present
- DNA analysis showed the possible presence of DNA from more than one person
- All of the confirmed components corresponded with the components in Pickering’s DNA profile
- The other components found matched the components of P’s DNA profile
R v Pickering (2012)
Vulva
- Not possible to determine if saliva was present
- DNA analysis showed the presence of DNA from more than one person
- All of the components could have originated from either ‘P’ or Pickering
R v Pickering (2012)
The vest
- Not possible to determine if saliva was present
- DNA analysis indicates the presence of DNA from at least three people
- All of the corresponding components from ‘P’ and Pickering were present
- There were components which could not have originated from ‘P’ or Pickering
R V Pickering (2012)
Prosecution scientist
- None of the DNA results were suitable for statistical analysis as there must have been at least one other unknown person who contributed to the mixed profile
- Based on the scenario that Pickering had licked P’s left breast and vagina the findings could entirely be explained by her account.
- On Pickering’s account findings were unlikely
- He concluded that the results were more likely to be consistent with P’s account than Pickering’s account but he could not express a view on the strength of support is no statistical evaluation could be carried out
R v Pickering (2012)
DNA evidence on the underwear
The DNA evidence relating to the knickers should not be admitted in terms of either:
* Amylase
* DNA
The inability to provide an objective strength of support skill opinion demonstrated that the conclusions were not scientific.
Pickerings defense appealed the conclusions of the DNA evidence
R v Pickering (2012)
Appeal conclusions on the admissibility of DNA evidence
- If all of the DNA evidence did was to show that the findings were consistent with ‘P’s’ account and that Pickering could not be excluded then its relevance was peripheral
- It was however prejudicial and confusing and should not have been admitted
- The case is solely on whether or not Pickering was a possible contributor and could not be excluded thus the contention that there was no statistical evidence or no evaluative evidence in the form of a hierarchy was not relevant
Admissibility
The quality of being acceptable or valid, especially as evidence in a court of law.
Inadmissible
Unable to be accepted in the court of law
R v Pickering (2012)
Overall conclusion
- That the presence of amylase in the knickers was strong supportive evidence of ‘P’s’ account.
- That account was also supported by evidence of recent complaint
- The and the evidence was relevant and it was properly admitted and correctly summed up to the jury
- The conviction is safe
Advancing Technology
- In addition to the routine application of enhanced sensitivity techniques, multiplexes in 2015 frequently achieve results from low quantities of DNA
- The incidence of complex mixtures and of low template profiles exhibiting stochastic effects is increasing and so the conditions in which subjective opinion tends to be relied on are more commonly encountered
- As a consequence, there is an increasing risk of cognitive contamination affecting DNA evidence.
Dlugosz (2011)
Convictions
- Convicted of burglary robbery and manslaughter & sentenced to imprisonment for public protection for seven years
- Subsequently retried and convicted of manslaughter & robbery and sentenced to 9 years in prison
Dlugosz (2011)
Details
- On 27th November 2008 an elderly lady’s house in North London was burgled
- On 1 January 2009 the victim was found dead in her house
- She had been tied up and being unable to free herself had died from hypothermia
- Her ankles and wrists had been bound with silver tape and her mouth had been gagged with the same tape to stop her screaming
- The house had been ransacked
Kube Dlugosz (2011)
Background
- Kuba Dlugosz, 33, of no fixed address, was wanted on a European arrest warrant for skipping prison in Poland when he attacked the 83-year-old at her £1million house in Leweston Place, where she had lived since childhood.
- The convicted robber fled to the UK in 2007
Fingerprints
Kube Dlugosz (2011)
Forensic Examination
- No fingerprints were found
- It was assumed the offenders had worn gloves
- The only forensic evidence was a very small quantity of DNA on two of the three chisels left at the premises
- The amounts were less than 200 pg.
- The results showed a mixed profile
- When arrested in July 2010 he answered no comment to all questions.
Kube Dlugosz (2011)
The case continutes
- Whilst on remand his telephone calls were recorded.
- In September 2010 his co-offender Wyrostek was arrested for the same offence.
- He had admitted his part in the burglary and admitted they put masking tape on the victims face.
Kube Dlugosz (2011)
Mistakes and bias
- Applied to exclude the DNA evidence
- His previous convictions were put before the jury
- He did not give evidence
Kube Dlugosz (2011)
DNA
- The quantity of DNA recovered from chisels was very low
- The largest yielded 119 pg.
- The trial focused on two samples from the chisels all found at the house
- Chisel two, the handle BRW/1/3
- Chisel two, the blade BRW/1/3
Kube Dlugosz (2011)
DNA results
- The crime sample was revealed it was a male and contained mixed profiles.
- Although the results could not be separated out to individual contributors, several of the alleles in the appellant’s DNA were represented as prominent components.
- The experts will also agreed that it was impossible to provide statistical/numerical assessment of the likelihood ratio
Kube Dlugosz (2011)
Defense proposition
- The Defence accepted that Dlugosz could have made a contribution to these samples.
- However the number of contributors to the DNA profiles could not be determined.
- It was impossible to resolve these mixed DNA profiles into individual profiles of the contributors.
- It was therefore not possible to calculate a robust statistic indicating the strength of the link between Dlugosz and DNA in the two samples.
- Although they provided an investigative lead they did not provide cogent evidence.
Kube Dlugosz (2011)
Prosecution case
The prosecution scientist, in an effort to be helpful expressed the view:
* “…To evaluate these results I have considered the following two propositions”
* DNA from Dlugosz was present on the swabs
* Or DNA from Dlugosz was not present on the swabs
* In my opinion, the DNA profile results provide support for the presence of DNA from Dlugosz on the swab but I am unable to quantify the level of this support
Kube Dlugosz (2011)
Exclusion of DNA evidence
The defence sought to exclude the evidence of DNA:
* Because without statistical evaluation
* Its significance could not be evaluated
* It should not be placed before the jury
Kube Dlugosz (2011)
Conclusions
- The jury heard the evidence of the two experts and were in a position to use that evidence in accordance with a clear and careful summing up bearing in mind the difference between the two experts was not that great.
- One saying it was rare and the other saying it was somewhat unusual
Kube Dlugosz (2011)
Safety of the conviction
- It was argued but there was a real danger that (a) the jury had taken weak DNA evidence and then been (b) prejudiced by the previous convictions of Dlugosz, particularly those that were 8½ years old.
- It seemed to the appeal that considering the totality of the evidence that the view of the experts on DNA that finding all of the individual’s alleles in a mixed sample of the kind analysed in the case was either “rare” or “somewhat unusual” was of real assistance.
- Taking the evidence of the telephone conversations there was, in the appeals judgement sufficient evidence of previous convictions to be properly open to consideration in determining the guilt. Taking all that evidence together and the judges careful, clear and fair summing up they saw no reason to consider the conviction unsafe
Defense
The rules of disclosure
- Within of England and Wales no prior disclosure of the defendant’s account is required
- This often means that the DNA scientist is required to make their own, suppositions about appropriate defence hypotheses.
Mitigation stratefies of bias
DNA
Currently deployed in the UK and overseas:
* Prior interpretation of the case sample result before the reference result is revealed
* Formally noting the following from the DNA result, prior to comparison with the reference profile
a. Suitability to include or exclude
b. Assessment of the number of contributors
c. Level of representation of contributors
d. Potential for stochastic effects
e. Identification of likely/unlikely genotype combinations that might explain the mixture
Repeats
Recommended for DNA profile interpretation in all circumstances
- Full checking via repeat interpretation by an experienced and competent colleague
- Including prior interpretation of the case sample result before the reference result is known
- The check should be conducted independent of and uninfluenced.
- The reporting scientist should use original unmodified hard copy or electronic results that are free from annotation.
case information
Careful selection
- Of case stains/samples for testing to minimise the occurrence of mixtures and low template issues.
- The selection should be informed by case information.
- Duplicate analyses to assess stochastic effects in low template samples
- Replication should be applied whenever a poor quality profile is to be relied on to progress an investigation or provide evidence against a suspect.
- It improves the scientist’s ability to gauge accurately whether or not the sample is suitable for any form of comparison or statistical evaluation.
Analysis and Interpretation
- Is carried out blind, in the complete absence of any information about the case
- This approach is practised in some jurisdictions and eliminates the risk of some types of bias
- It does present the practical challenge of separating
- Case strategy
- Hypotheses testing
- Stain selection, etc.
- From result interpretation and reporting in the context of the case
What should be done when a numerical evaluation is not possible?
- When a numerical evaluation is not possible, it remains of crucial importance that qualitative & subjective judgements of pertinent profile features and their combined likelihood are assessed under the hypotheses framed by both the prosecution hypothesis and defence hypothesis separately.
- The final opinion of evidential weight must be based on how much, if any, comparison of separate assessments favours one hypothesis over the other, as with a likelihood ratio (LR).
A completely blind checker
- Repeats the full interpretation described in previous tests but in the absence of any contextual information relating to the case.
- This may present practical challenges, particularly within small organisations.
- But may provides assurance for the courts that the interpretation is free from contextual bias.
Objective Evaluation
If there is no suitable option for objective evaluation, only employ qualitative and subjective-based approaches that:
* Have been validated
* And therefore have demonstrated robustness of resultant conclusions and opinions
Such procedures should include system performance data indicating when the approach breaks down and is no longer valid
All supported by training and education
Scottish Police Services Authority
Summary of findings
- Improved note taking, including a demonstration of features used in identification.
- A complex marks process to manage variance in opinion between examiners.
- Those involved in the technical review process have no prior knowledge or access to case-related information or the technical findings of any other examiners.
- A blind verification process for lead identifications in which verifying examiners have no knowledge of the technical findings of any previous examiners.
- The removal of any case context information or related communication documentation from the verification process in any circumstances.
- Regular dip-sampling of all completed casework.
- Training programmes for examiners exploring cognitive bias and its impact on the human decision-making process.
ACE-V
- ACE-V is the most commonly accepted approach to fingerprint comparison in the UK and the USA.
- An examiner analyses a mark.
- The examiner than compares the mark with a known print.
- Having compared the images, the examiner evaluates what they have seen and reaches a decision and the results are then subject to vertification by one or more addtional examiners.
What forms of bias can be introduced during fingerprint examination?
- Contextual
- Confirmation
- Cultural
What can impact a fingerprint examiners decision making process?
- Nature and details of the crime including background information
- Association with or personal knowledge of the victim or their circumstances
- Status of suspects or person(s) already in custody for the crime
- Previous criminal activity of suspects or persons of interest
- Location of the crime (an area close to their home)
- Media or public interest associated with the crime
- Personal moral codes or behaviours
- Time pressure from investigating officers or office managers
Confirmation bias
Fingerprints
- Police Officers or prosecution services, with both hoping that the examination outcomes will help to ‘solve the case’ or ‘secure a conviction’.
- Contributing to the detection of crime is considered a fundamental aspect of fingerprint bureau service delivery
- Personal identification or ‘hit’ rates are used as key performance indicators at both organisational and individual level.
- A personal moral code to ‘do the right thing
- Having prior knowledge of the previous examiner’s findings and conclusions may also expose fingerprint examiners to the risk of confirmation bias.
- Having found a number of features from an unknown impression to agree with features in an impression from a known source, the examiner will then begin to reason backward, finding features in the unknown impression that are suggested by those in the known print.
Culturaal bias
Fingerprints
Cultural influence that can impact on the decision-making process include:
- Strict hierarchical structures based on time served rather than competence
- Over confidence in individual or organisational competence
- Lack of interaction with peers or exposure to alternative methods of working
- Lack of acceptance of the potential for errors or effective root cause analysis of errors.
What are two examples when cognitive bias has become an issue?
- Brandon Myafield 2006
- Shirley Mckie 1999
Federal Bureau of investigation
Brandon Mayfield
- Following the procedure review instigated as a result of the Brandon Mayfield case, the FBI introduced a system of blind verification
- The independent application of Analysis, Comparison, and Evaluation (ACE) by another qualified examiner who does not know the conclusions of the primary examiner
When does blind verification take place?
Fingerprints
- Those with a single mark conclusion
- Circumstances where there are conflicts between examiners
- Circumstances where there are conflicts on decisions of ‘value’ or ‘no value’
Mitigating bias in fingerprint analysis
Part 1
- Introduce control mechanisms for consideration with regard to fingerprint examination.
- Survey and breakdown the extent of current contextual information available to examiners and assess the added value that each piece of information brings to the examination process.
- Remove or limit contextual information that adds no tangible value to the fingerprint examination process.
- Remove or limit contextual information made available to verifying or subsequent examiners.
Mitigating bias in fingerprint analysis
Part 2
- Introduce a blind verification process for identified casework assessed as at greatest risk
Introduce a blind element to a technical review process for analyses, comparisons and/or evaluations that are considered complex or cause a variance in opinion between examiners - As part of a technical review process for complex marks or circumstances where examiners have a variance in opinion, introduce an appropriate and proportionate note-taking strategy, which requires examiners to provide written and visual accounts of their reasoning and findings
Mitigating bias in fingerprint analysis
Part 3
- Develop bespoke training programmes to raise awareness of the cognitive issues
- As part of an established quality management system, instigate an effective review and monitoring process to provide assurance that the risk treatment and control measures continue to provide effective risk management.
The generic forensic process encompasses the interpretation and reporting of comparison of ‘marks’ cases.
a. Examine the item/mark recovered from the crime scene
b. Use recovery and enhancement techniques as required
c. Generate/examine the ‘control’ item
d. Make test marks if required in the appropriate manner
e. Undertake a comparison using appropriate methods and equipment.
f. Interpret and evaluate findings
g. Verify the result
h. Describe findings in a statement or report
i. The scientist may be called to court to give oral testimony
Mark comparison seeks to establish…
- If a ‘mark’ (the unknown)
- Has been made by the submitted exhibit (the known)
- Or has been made by the same item, for example, a revolver that has not been recovered could be responsible for discharging multiple bullets.
Mark comparison is based on..
- The comparison of detail and is therefore observational.
- Any examination is dependent on the visual quality and clarity of the detail that is observed by the examiner.
- Process of pattern recognition.
In mark comparison, the scientist is looking to…
- Determine if the detail present in the mark
- Matches characteristic detail on the item
- Or in a test mark
- Or is significantly different
- An assessment of what the detail is and how it has been produced must consider general characteristics common to a set of items.
Mark comparison
Level of significance
- The final assessor of the level of significance of any agreement between the marks is the human operator; there is no significant instrumental analysis.
- In footwear mark comparisons, the methods are normally side-by-side comparisons or overlay.
- In this way the footwear expert assesses the level of agreement in terms of the pattern, pattern configuration, mould/moulding detail, wear and damage.
- The assessment is subjective, although reference material and data can be used to support the evaluation of the findings.
Mark comparison
Spectrum of bias risk
Detail: Low risk
The detail in the marks is clear, well defined and unambiguous.
Mark comparison
Spectrum of bias
Detail: High risk
Marks are confused and complex, of poor wuality and the detail present is poorly defined
Mark comparison
Spectrum of bias
Equipment: Low risk
Optimum visualisation of the detail in a mark using appropriate equipment and enhancement techniques.
Mark comparison
Spectrum of bias
Equipment: High risk
Poor or inappropriate equipment and enhancement techniques.
Mark Comparisons
Spectrum of bias
Approach/examiner: Low risk
- There is a methodical approach with defined standards built on principles that have been tested and validated.
- Possible confirmation bias may reduce as a consequence of the comparison reviewer having less contextual information.
Mark comparison
Spectrum of bias
Approach/examiner: High risk
- When the approach is unresearched and personal to the operator.
- When the expectation of an ipen case file hit is very low.
Mark comparison
Spectrum of bias
Scientist/examiner: Low risk
Scientists/examiners are well trained, expereinced and continuously meet acceptable standards of competence.
Mark comparison
Spectrum of bias
Scientist/examiner: High risk
Scientist/examiner are inexperienceed, unmonitored and left to adopt their own approach.
Mark comparisons
Bias mitigation strategies
Part 1
- Comparison of expected, pre-assessed outcomes of appropriate hypotheses with actual results.
- Full disclosure of all data used in the evaluation.
- In all firearms classification cases, the reviewer should clearly set out what is official guidance and what is statute, ensuring that alternative classification hypotheses are addressed to counter any confirmation bias
- Use a completely ‘blind’ checker who repeats the full interpretation, but in the absence of any contextual information relating to the case.
- An acceptable alternative is that the result will be subject to a critical findings check by a second authorised examiner. The checker then undertakes a detailed independent review wherever possible without knowledge of the previous practitioner’s conclusion.
Mark comparisons
Bias mitigation strategies
Part 2
- Validation testing of qualitative and subjective-based approaches to demonstrate the robustness of conclusions and opinions
- Development of standards and quality managed procedures for qualitative and subjective-based methods, including system performance data indicating when the approach breaks down and is no longer valid
- Practitioner training in the specific method used, together with initial and ongoing competency assessment.
- Training and education in relation to the risks of cognitive bias in firearms classification and marks’ comparison generally.
- An approach to quality that includes the assessment and ongoing monitoring of the competence of practitioners including the use of proficiency tests, declared and undeclared trials.
- Providers should ensure that a validated form of context management is applied.
- The use of blind trials should be introduced to increase the ‘success’ rate of cold hits.
Particulate trace evidence
Including hair and fibre
- Contact trace evidence includes a wide range of materials.
- Including particulate material such as glass, paint, hairs and fibres.
- However, whilst the range of particulate trace materials is wide, the analysis of such material essentially follows the same process, which involves the comparison of crime (unknown - recovered) material with one or more known/reference samples.
Recovery of particulare trace materials using appropriate techniques.
- The examiner is often presented with a large amount of debris that potentially contains some of the target material and other materials which have no relevance to the case.
- The material can then be compared with the relevant reference sample(s) using the appropriate microscopy and instrumental/analytical techniques.
- It is necessary to review the reference sample (s) and use features to enable an initial search of the recovered material to locate that which is of potential interest.
- This material can be recovered for more detailed comparison. using the appropriate microscopy & instrumental - analytical techniques.
- Scientific findings are evaluated and interpreted within the context of the case-specific information available (may be at source and/or activity level as appropriate).
- A report or statement describing the findings and providing an opinion on their significance is then provided.
Absence of matching fibres concluded as being neutral
Examples of possible cognitive bias
- The examination of car seat tapings for a transfer of fibres from the clothing of an individual who is alleged to have stolen and driven the car for some hours results in no matching fibres being found.
- The defendant has made no comment
- In this situation, it is tempting to conclude that the absence of matching fibres is neutral and does not assist in addressing whether or not the individual had been in the car.
- However, if the information available provides no explanation for the absence of matching fibres (for example, the defendant might have had time to change clothing before arrest) and the scientist had a high expectation of finding matching fibres if the contact had occurred as alleged, the absence of matching fibres may well support the view that the defendant had not been in the car.
Difference in treatment of crime and reference matierial post transfer?
- A fibre examiner faces considerable difficulty in dealing with cases where clothing has been altered at a chemical level in the period between the offence and seizure of the clothing
- For example, where the body of a victim has been submerged in a river or at sea for some time, causing the dye in the clothing to fade
- In this situation, the challenge for a fibre. examiner is firstly searching for fibres without a reference sample that is representative of the fabric at the time of the offence.
Bias mitigation strategies
Trace
- Independent checking – where only subjective observational assessments of a match are possible.
- Independent checking of analytical results – where instrumental techniques are used.
- Use of statistical approach to evaluation – A statistical approach can be applied rather than relying on the experience of the practitioner.
- Case assessment and interpretation – a robust and documented comparison of expected, pre-assessed outcomes with actual results under appropriate competing hypotheses.
- Training – appropriate training of practitioners in the methods employed can demonstrate initial and ongoing competence.
- Quality assurance trials
Video and audio examination process
- Recovery of video, photo or audio material related to the crime scene.
- Items are received by the analyst along with relevant case information and the questions to be addressed by the scientific work
- Generation of an exact copy of the original, then use of techniques as required to clarify or clean up the copy of the image or audio signal.
- Examination of the copied material recovered from the crime scene and notation of features determined to be reliable
- Examination of the ‘control’ item.
- Undertaking a comparison using appropriate methods and equipment.
- Interpreting and evaluating findings
- Verification of the result.
- Findings are described in a statement or report.
- The scientist may be called to court to give oral testimony.
The taxonomy
Level 1 - The trace evidence
Level 2 - Reference materials
Level 3 - The case information
Level 4 - Base rate expectations that arise from their experience (e.g., when the examiner expects a particular result.
Level 5 - Organizational and cultural factors.
Case managers
Taxonomy
- A straightforward tool for dealing with bias from case information (Level 3)
- Designed to prevent contextual bias by protecting the examiner from exposure to task-irrelevant information.
Bias v Task Relevance
Taxonomy
- Some types of information, while potentially biasing, may also be task relevant (5).
- For example, in some instances, evidence that analysts must examine, to perform their duties may contain information that is potentially biasing.
- For example, cases in which Level 1 information, the trace evidence being evaluated, contains contextual information (e.g., blood spatter patterns that contain information about the nature of the crime, or handwriting and voice samples in which the meaning of the words is potentially biasing)
Reference samples
Taxonomy
- Another example of relevant material that is also potentially biasing
- These samples are clearly relevant because the analyst must compare them to trace evidence samples to determine whether they are similar enough to conclude that they come from the same source.
- But it is possible that an analysts interpretation of the trace evidence might inadvertently be influenced by knowing the characteristics of the reference samples.
Linear sequential unmasking
This approach is linear in the sense that one must begin with the trace evidence before being exposed to and working with the reference material, thus working from the evidence to the suspect, rather than from the suspect to the evidence.
Linear sequential unmaksing
What does it mandate?
- This approach does not prevent exposure to biasing relevant information.
- However, it does mandate that this information be presented as late as possible in the examination process and only when it is necessary.
Linear sequential unmasking
Critical element
A critical element of sequential unmasking is that the forensic examiner must first examine and document the trace evidence from the crime scene (Level 1), before being exposed to the known reference material (Level 2).
Trace materials
- The trace evidence should be interpreted the same in this initial step, regardless of any suspect who is considered as a possible (or even likely) source of the trace evidence.
- However, according to sequential unmasking examiners are permitted to revisit, as well as change, their initial analysis of the trace evidence once they have reviewed the reference material, provided that they document these changes.
Taxonomy of seven sources of factors that may unconsciously effect the decision making of forensic experts.
Dror 2017
- Cogntive architecture and the brain
- Training and motivation
- Organizational factors
- Base rate expectations
- Irrelevant case information
- Reference materials
- Case evidence
What are the 3 categories in Drors factors affecting decision making?
- Human nature
- Environmental culture & Experience
- Case specific
Contextual buas
Policy briefing summary
When irrelevant context information about an event or the way in which the information is presented influences reasoning
Confirmation bias
Policy briefing summary
Interpreting information or looking for new evidence in a way which conforms to pre-existing beliefs or assumptions.
Expected frequency bias
Policy briefing summary
Being accustomed to a particular result occurring at a certain rate and then expecting it to keep occurring at the same rate.
How far back does cognitive bias go?
As far back as 2009, NAS report criticized forensic scientists for making insufficient efforts to reduce their vulnerability to cognitive and contextual bias.
What have The Organization of Scientific Area Committees (OSAC),as well as the National Commission on Forensic Science created?
The Organization of Scientific Area Committees (OSAC),as well as the National Commission on Forensic Science, have created committees on human factors that are specifically charged with examining this issue of bias.
