Topic 7 - Bias Flashcards
A key requirement of the Forensic Science Regulator’s Codes of Practice and Conduct for Forensic Science Providers and Practitioners (the Codes) is that they act with…
- Honesty
- Integrity
- Objectivity
- Impartiality
Categories of cognitive bias
- Experimental bias
- Confirmation bias
- Anchoring effects or focalism
Expectation bias
Where the expectation of what an individual will find affects what is actually found.
Confirmation bias
- Closely related to expectation bias
- Whereby people test hypotheses by looking for confirming evidence rather than for potentially conflicting evidence
Anchoring effects of focalism
Occur when an individual relies too heavily on an initial piece of information when making subsequent judgements, which are then interpreted on the basis of the anchor.
Causes of bias?
- Overconfidence
- Recency
- Salience
Overconfidence
Causes of bias
- Some of us are too confident about our abilities and this causes us to take greater risks.
- Experts are more prone to this bias than laypeople since they are more convinced that they are right
Receny
Causes of bias
The tendency to weigh the latest information more heavily than older data
Salience
Cause of bias
Our tendency to focus on the most easily recognisable features of the situation or concept
Availability heuristic
People who overestimate the importance of information that is available to them.
Bandwagon effect
The probability of one person adopting a belief increases based on the number of people who hold that belief.
Blind spot bias
Failing to recognise your own cognitive biases is a bias in itself
Choice supportive bias
When one chooses something which we tend to feel positive about
Clustering illusion
Tendency to see patterns in random events.
Confirmation bias
We tend to listen only to information that confirms our preconceptions.
Conservatism bias
Where people favour prior evidence over new evidence or information that has emerged
Information bias
- The tendency to seek information when it does not affect action.
- More information is not always better
Ostrich effect
The decision to ignore a dangerous or negative information by burying one’s head in the sand.
Outcome bias
Judging decisions are based on the outcome rather than how exactly the decision was made.
Selective perception
Allowing our expectations to influence how we perceive events.
Survivorship bias
An error that comes from focusing only on surviving examples causing us to misjudge a situation.
Criminal procedure rules - Part 19
An experts duty to the court includes the following:
* An expert must help the court to achieve the overriding objective – by giving an opinion that is objective, unbiased and wihtin the experts area of expertise.
* This duty overrides any obligation to the person from whom the expert receives instructions or by whom the expert is paid
The role of the forensic expert
- To evaluate scientific findings and the results of analytical tests in the context of the relevant case circumstances.
What criteria should the exprts opinion meet?
An expert opinion should meet the following criteria
* Balanced
* Robust
* Logical
* Transparent
Categories of cognitive bias
Effects
- Contexttual bias
- Role effects
- Motivational bias
- Reconstructive effects
Contextual bias
Cognitive bias
Where someone has other information aside from that being considered, which influences (either consciously or subconsciously) the outcome of the consideration.
Role effects
Cognitive bias
- Are where scientists identify themselves within adversarial judicial systems as part of either the prosecution or defence teams.
- This may introduce subconscious bias that can influence decisions, especially where some ambiguity exists.
Motivational bias
Cognitive bias
Occurs where, for example, motivational influence on decision making, results in information consistent with a favoured conclusion tending to be subject to a lower level of scrutiny than information that may support a less favoured outcome.
Reconstructive bias
Cognitive bias
- Can occur when people rely on memory rather than taking contemporaneous notes.
- In this case people tend subsequently to fill in gaps with what they believe should have happened.
- And so may be influenced by protocol requirements when recalling events some time later from memory.
Risks are bias are lower when…
Risks are bias are lower when results are clear and unambigous.
Risks of bias are greater when…
Results
Risks of bias are greater when results are:
* Complex
* Poor quality
* Increased reliance on subjective opinion
Risks of bias are lower when..
Approach
Risks are lower when there is a methodical approach with defined standards built on principles that have been tested and validated.
Risks of bias are greater when…
Approaches
When the approach is unresearched and personal to the practitioner.
Risks of bias are lower when…
Practitioners
When practitioners and checkers are:
* well trained
* Experienced
* Continuously meet acceptable standards of competence
Risks of bias are higher when…
Practitioner
When practitioners and checkers are:
* inexperienced
* Unmonitored
* Left to adopt their own approach
Risks of bias are lower when…
Checking
- When interpretation is checked by a competent peer.
- who conducts a separate interpretation fully independent and without influence from the reporting scientist.
Risks of bias are higher when…
Checking
When checking is less rigorous and or conducted collaboratively.
Debias
The reduction or elimination of the impact of bias in decision making and problem solving.
Photogrammetry
The practice of obtaining reliable information about physical objects through the processes of recording measuring and interpreting photographic images.
Psychological contamination
Exposure to other information that is irrelevant to the assessment but that introduces sub conscious bias into the findings.
Reconstructive effects
The tendency when people rely on memory to fill in gaps on recall with what they believe should have happened.
Threats to impartiality
- Being the sole reviewer of their critical findings
- Being over-familiar with or trusting another person instead of relying on objective evidence
- Having organisational and management structures that could be perceived to reward, encourage or support bias where
- A culture of performance measurement
- And time pressures could potentially pressurise examiners into biasing decisions.
Why is it important to have the whole picture?
- Important for constructing and testing relevant hypotheses and propositions.
- However, if the whole case file is handed over to an analyst with all the detail, there is the perception that bias could have occurred and may be open to challenge in court
How can suseptibility to psychological and cognitive influences be assessed?
- This may be achieved through a proficiency-testing programme
- Utilising mocked up casework samples for which the expected outcomes of testing and evaluation are known.
- Whilst blind trials are the gold standard in providing the most reliable indicator of real-life performance, in reality they can be very time-consuming.
- Good practice adopted by many laboratories is to undertake a mixed programme of both declared and undeclared trials, with the proficiency of all individuals tested on a regular basis
Awareness, training and competence assessment
- Practitioners need training in how the evaluation procedures assist in overcoming some of these risks in their respective roles, as well as what residual risk may remain.
- Given that susceptibility to psychological and cognitive influences varies between individuals, there may be merit in assessing these susceptibilities as part of the recruitment or selection procedures for new staff.
Avoiding reconstructive effects
- The taking of contemporaneous notes or technical records is another stipulation in the Codes (section 15.2.3)
- Adherence with this requirement wherever it is practicable to do so at all stages in the collection and processing of forensic evidence provides the best safeguard against potential reconstructive effects .
Avoiding role effects
- This is where scientists are subconsciously influenced by acting on behalf of the defence or prosecution.
- Difficult to demonstrably eliminate given the adversarial nature of the CJS within the UK.
- These pressures can also apply in the case of police laboratories in providing services to an internal customer.
Forensic Process outline
- Define the requirement
- Develop an examination strategy
- Agree the examination strategy with the client
- Lead scientist instructs analyst to carry out forensic examinations and analyses
- Review the quality and content of examination results
- Compare the results with the reference samples and marks.
- Evaluate and interpret the scientific findings and analytical tests
- Second expert verifies the findings
- Communicate the scientific findings and analytical tests
During the forensic outline, what responsibility does the expert have?
Reveal
It is the responsibility of the expert to:
* Reveal their work (disclose)
* Record all information received
* Record details of interpretation
When working in the evaluative mode, the expert goes through:
When working in the evaluative mode, the expert goes through:
* A formal process of pre-assessing the expected probabilities
* For a realistic range of possible outcomes
* In as many or as few categories as is sensible for the examination
* Recording their opinions
Realistc list of outcomes
- At source level, and certainly at activity level, it may be that each category in the realistic list of outcomes is considered firstly under the assumption that the prosecution hypothesis is true, and secondly under the assumption that the defence hypothesis is true.
- These are used to provide an expected outcome that may be either qualitative or quantitative with the latter expressed as a likelihood ratio.
Recommended good practise
- Identify whether the scientist’s role in the case is investigative (for example, intelligence) or evaluative (judicial)
- Seek clarity on what are the issues, the purpose and how this fits into the hierarchy of sub-source (for example, touch DNA), source, activity and offence level propositions.
Develope an evaluative examination strategy
- Formulate relevant prosecution and defence alternatives based on the case circumstances and information provided.
- Consider any agreed assumptions that are used in formulating these alternatives.
- Use assessment of possible outcomes to determine which tests are most informative and discriminating.
- Use this pre-assessment to assign a weight to an exhaustive list of possible outcomes, giving the expected outcome for each, expressed as a LR where these are quantitative.
- This approach provides clarity on the alternatives being considered, and the pre-assessment of weight for all outcomes avoids the potential bias of using the observed results to assign weight of evidence.
Carry out forensic examinations and analyses
- Review the quality and content of examination results.
- Decisions on the suitability of the results and marks for later comparison are made at this stage, to avoid post-comparison rationalisation of opinion on quality.
- Compare the results with the reference samples and marks.
- The quality and suitability of the questioned result has already been assessed so this is not influenced by the reference result.
Evaluative and Interpret scientific findings and analytical tests
- Mitigate the confirmation bias by using the LR or qualitative expectation that was assigned to each outcome before the examinations and tests were performed
- Pre-assessment enables the scientist to explain how the weight of the evidence has been assigned
- Provide details of the assumptions that have been made
- Give the basis of the expert opinion and specify the propositions considered, with the reasoning for these, based on the case context
- Include any limitation of the opinion
- Where there is a range of opinion on the matters dealt with in the report, the expert’s report must summarise the range of opinion and give reasons for the expert’s own opinion as required by the Criminal Procedure Rules
Recommended Good Practise
Second expert
- Verification by a second expert - Independent review at this stage in advance of communicating the result to the client
- Communicate the scientific findings and analytical tests
The police response to a reported crime requires many factors to be taken into consideration:
The police response to a reported crime requires many factors to be taken into consideration:
* Preserving the scene
* Securing evidence
* The speed of response
* The proportionate use of resources based on the seriousness of the crime
* All are overridden by the most pressing priority -the preservation of life
Crime scene activities and risk of bias impact
The potential impact may be significant, for example:
* It could result in failure to secure the required evidence if a crime scene.
* An investigation could be closed prematurely, resulting in crucial evidence being lost.
* It could mislead an investigation by investigators focusing too early and incorrectly on a false lead, and other evidence is potentially overlooked.
* If undertaken incorrectly activities could result in ‘psychological contamination’ of evidence downstream in the forensic analysis and interpretation processes.
Oppoertunities for cognitive bias
Gathering information prior to scene attendance.
* Prior to scene attendance information is gathered from any available source regarding the incident to be investigated.
* This may include witness or victim accounts as to what is alleged to have happened and by their nature these may.
The cognitive processess entail…
The cognitive processess entail:
* Determining locations and boundaries of the scene
* The entry/exit points of the offender
* Information received and inferences
What factors can cause bias in the cognitive process?
- The past experiences of an individual on which they may base their decisions are subjective.
- Convenience may be more relevant and have more impact in real life. - For example, establishing the boundary by taping from lamp post to lamp post is commonplace simply because they are already there.