Bias Flashcards
1
Q
What is the connection between CAI and bias?
A
- important to have a transparent and logical process for bother assessment and interpretation
- we need info to determine LR and need info to ascend hierarchy of propositions
- information conveyed incorrectly may bias opinion
- CAI requires choices in both components and if conducted incorrectly then these choices may introduce bias
2
Q
What is a key requirement of the forensic science regulators codes of practice and conduct for forensic science providers and practitioners (the codes) ?
A
- is that they act with honesty, integrity, objectivity, impartiality
3
Q
What are seven types of categories of cognitive bias
A
1 - expectation bias
2 - confirmation bias
3 - anchoring effects
4 - contextual bias
5 - role effects
6 - motivational bias
7 - reconstructive effects
4
Q
expectation bias
A
- experimenters bias
- where expectation of what an individual will find affects what is actually found
5
Q
confirmation bias
A
- closely related to expectation bias
- whereby people test hypotheses by looking for confirming evidence rather than for potentially conflicting evidence
6
Q
anchoring effects
A
- focalism
- occurs when an individual relies too heavily on an initial piece of information when making subsequent judgements, which are then interpreted on the basis of the anchor
7
Q
contextual bias
A
- where someone has other information aside from that being considered, which influences (either consciously or subconsciously) the outcome of the consideration
8
Q
role effects
A
- where scientists identify themselves within adversarial judicial systems as part of either prosecution/defence teams
- this may introduce subconscious bias that can influence decisions, especially where some ambiguity exists
9
Q
motivational bias
A
- occurs where for example motivational influence on decision making results in information consistent with a favoured conclusion tending to be subject to a lower level of scrutiny than information that may support a less favoured outcome
10
Q
reconstructive effects
A
- can occur when people rely on memory rather than taking contemporaneous notes
- in this case people tend subsequently to fill in gaps with what they believe should have happened
- so may be influenced by protocol requirements when recalling events some time later from memory
11
Q
What are 14 other types of bias?
A
1 - overconfidence
2 - recency
3 - salience
4 - availability heuristic
5 - bandwagon effect (groupthink)
6 - blind spot bias
7 - choice supportive bias
8 - clustering illusion
9 - conservatism bias
10 - information bias
11 - ostrich effect
12 - outcome bias
13 - selective perception
14 - survivorship bias
12
Q
overconfidence
A
- some of us are too confident about our abilities and this causes us to take greater risks
- experts are more prone to this bias than laypeople since they are more convinced that they are right
13
Q
recency
A
- tendency to weigh the latest information more heavily than older data
14
Q
salience
A
- tendency to focus on easily recognisable features of the situation or concept
15
Q
availability bias
A
- people who overestimate the important of information available to them
16
Q
bandwagon effect (groupthink)
A
- probability of one person adopting a belief increases based on number of people who hold that belief
17
Q
blind spot bias
A
- failing to recognise your own cognitive biases is a bias in itself
18
Q
choice supportive bias
A
- when one chooses something which we tend to feel positive about
19
Q
clustering illusion
A
- tendency to see patterns in random events
20
Q
conservativism bias
A
- where people favour prior evidence over new evidence or information that has emerged
21
Q
information bias
A
- tendency to seek information when it does not affect action
- more information is not always better
22
Q
ostrich effect
A
- decision to ignore a dangerous or negative information by burying ones head in the sand
23
Q
outcome bias
A
- judging decisions are based on the outcome rather than exactly how the decision was made
24
Q
selective perception
A
- allowing out expectations to influence how we perceive events
25
Q
survivorship bias
A
- an error that comes from focusing only on surviving examples causing us to misjudge a situation
26
Q
What does part 19 of CPR say?
A
- that an experts duty to the court includes the following:
- an expert must help the court to achieve the overriding objective by giving an opinion that is objective and unbiased and within the expert’s area or areas of expertise
- this duty overrides any obligation to the person from whom the expert receives instructions or by whom the expert is paid
- used to be part 33
27
Q
What is the role of a forensic expert?
A
- to evaluate scientific findings and the results of analytical tests in the context of the relevant case circumstances
28
Q
What criteria should expert opinion meet?
A
- balanced
- robust
- logical
- transparent
29
Q
When are risks of bias lower
- results
- approach
- practitioners
- checking
A
- when results are clear and unambiguous
- a methodical approach
- with defined standards
- built on principles that have been tested and validated
- when practitioners and checkers are well trained, experienced and continuously meet acceptable standards of competence
- when interpretation is checked by a competent peer who conducts a separate interpretation fully independent and without influence from the reporting scientist
30
Q
When are risks of bias greater
- results
- approach
- practitioners
- checking
A
- when results are:
- complex
- of poor quality
- and there is an increased reliance on subjective opinion
- when the approach is unresearched and personal to practitioner
- when practitioners and checkers are inexperienced, unmonitored and left to adopt their own approach
- when checking is less rigorous and/or conducted collaboratively
31
Q
debias
A
- reduction or elimination of impact of bias in decision making and problem solving
32
Q
photogrammetry
A
- practice of obtaining reliable information about physical objects through the processes of recording measuring and interpreting photographic images
33
Q
psychological contamination
A
- exposure to other information that is irrelevant to the assessment but that introduces subconscious bias into findings
34
Q
what are three threats to impartiality
A
1 - being the sole reviewer of their critical findings
2 - being over-familiar with or trusting another person instead of relying on objective evidence
3 - having organisational and management structures that could be perceived to reward, encourage or support bias (where for example a culture of performance measurement and time pressures could potentially pressurise examiners into biasing decisions)
35
Q
Is it important to have a complete picture?
A
- often vital for constructing and testing relevant hypotheses and propositions
- however if knowing about certain aspects are assessed to work against the objective process in a particular method
- if whole case file is handed over to an analyst with all extraneous detail then even if there is no perceptible bias there is the perception that it could have occurred and may be open to challenge in court
36
Q
What does the susceptibility to psychological and cognitive influences vary between?
A
- varies between individuals
37
Q
How is susceptibility to psychological and cognitive influences measured?
A
- through a proficiency-testing programme
- utilising mocked up casework samples for which the expected outcomes of testing and evaluation are known
- blind trials are gold standard in providing most reliable indicator of real-life performance but in reality they can be very time-consuming
- good practice is to undertake a mixed programme of both declared and undeclared trials, with the proficiency of all individuals tested on a regular basis
38
Q
Awareness, training and competence assessment
A
- practitioners need training in how the evaluation procedures assist in overcoming some of these risks in their respective roles, as well as what residual risk may remain
- ideally this training should be practical
- given that susceptibility to psychological and cognitive influences varies between individuals, there may be merit in assessing these susceptibilities as part of the recruitment or selection procedures for new staff
39
Q
How are reconstructive effects avoided?
A
- by taking contemp notes or technical records
- accordance with this requirement wherever it is practicable to do so at all stages in the collection and processing of forensic evidence provides the best safeguard against potential reconstructive effects
40
Q
How are role effects avoided?
A
- where scientists are subconsciously influenced by acting on behalf of the defence or prosecution
- difficult to demonstrably eliminate given the adversarial nature of the CJS within the UK
- potentially compounded by the pressures of a commercial market in which a supplier/customer relationship
- these pressures apply whether an FSP is providing contracted services to the prosecution or defence or in the case of police laboratories in providing services to an internal customer
41
Q
What is 9 step process outline of forensic process within lab?
A
1 - define requirement
2 - develop an examination strategy
3 - agree examination strategy with the client
4 - lead scientist instructs analyst to carry out forensic examinations and analyses
5 - review quality and content of examination results
6 - compare results with reference samples and marks
7 - evaluate and interpret the scientific findings and analytical tests
8 - second expert verifies the findings
9 - communicate scientific findings and analytical tests
42
Q
What is the responsibility of the expert to do during this process?
A
- to record, retain and reveal their work (disclose)
- including the following
- record all info received and record details of interpretation
43
Q
What does expert go through when working in evaluative mode?
What can occur at source level?
What can occur at both source and activity level?
A
- a formal process of pre-assessing the expected probabilities for a realistic range of possible outcome in as many or as few categories as is sensible for the examination
- recording their opinions
- may be that possible outcomes could be simply the two hypotheses that the items being considered either match or do not match
- it may be that each category in the realistic list of outcomes is considered firstly under the assumption that the prosecution hypothesis is true, and secondly under the assumption that the defence hypothesis is true
- these are used to provide an expected outcome that may be either qualitative or quantitative (LR)
44
Q
What is recommended good practice for defining requirement
A
- identify whether the scientist’s role in the case is investigative (for example, intelligence) or evaluative (judicial)
- seek clarity on what are the issues, the purpose and how this fits into the hierarchy of sub-source (for example, touch DNA), source, activity and offence level propositions
45
Q
What is recommended good practice for developing an evaluative examination strategy
A
- formulate relevant prosecution and defence alternatives based on the case circumstances and information provided
- consider any agreed assumptions that are used in formulating these alternatives
- use assessment of possible outcomes to determine which tests are most informative and discriminating
- use this pre-assessment to assign a weight to an exhaustive list of possible outcomes, giving the expected outcome for each, expressed as a LR where these are quantitative
- this approach provides clarity on the alternatives being considered, and the pre-assessment of weight for all outcomes avoids the potential bias of using the observed results to assign weight of evidence
46
Q
What is recommended good practice for carrying out forensic examinations and analyses
A
- review the quality and content of examination results
- decisions on suitability of the results and marks for later comparison are made at this stage, to avoid post-comparison rationalisation of opinion on quality
- compare the results with the reference samples and marks
- quality and suitability of the questioned result has already been assessed so this is not influenced by the reference result
47
Q
what is recommended good practice for evaluating and interpreting scientific findings and analytical tests?
A
- mitigate the confirmation bias by using the LR or qualitative expectation that was assigned to each outcome before the examinations and tests were performed
- pre-assessment enables the scientist to explain how the weight of the evidence has been assigned
- provide details of the assumptions that have been made
- give the basis of the expert opinion and specify the propositions considered, with the reasoning for these, based on the case context
- include any limitation of the opinion
- where there is a range of opinion on the matters dealt with in the report, the expert’s report must summarise the range of opinion and give reasons for the expert’s own opinion as required by the Criminal Procedure Rules
- verification by a second expert (independent review at this stage in advance of communicating the result to the client)
- communicate the scientific findings and analytical tests
48
Q
What factors must be taken into consideration in the police response to a reported crime?
A
- preserving scene
- securing evidence
- speed of response
- proportionate use of resources based on seriousness of crime
- all are overridden by most pressing priority - preservation of life
49
Q
What is correlation between crime scene activities and risk of bias?
A
- could result in failure to secure required evidence if a crime scene investigation is closed prematurely resulting in crucial evidence being lost
- could mislead an investigation by investigators focusing too early and incorrectly on a false lead and other evidence is potentially overlooked
- if undertaken incorrectly activities could result in psychological contamination of evidence downstream in forensic analysis and interpretation process
50
Q
When are opportunities for cognitive bias
A
- gathering information prior to scene attendance
- prior to scene attendance information is gathered from any available source regarding the incident to be investigated
- this may include witness or victim accounts as to what is alleged to have happened and by their nature these may be consciously or subconsciously biased
51
Q
What are four cognitive processes at a scene?
Why might there be scope for bias to affect these decisions?
A
- determining locations and boundaries of the scene
- the entry/exit points of the offender based on observations
- information received and inferences
- past experiences of an individual on which they may base their decisions are subjective
- other factors such as convenience may be more relevant and have more impact in real life e.g. establishing boundary by taping from lamp post to lamp post as they are already there
52
Q
What are cognitive processes for creating a record of the scene?
Why might there be scope for bias to affect these decisions?
A
- selection of equipment
- decisions on which images to capture
- entails an assessment of the current case needs plus some anticipation of future needs
- depending on how the written record is crafted, there is a risk that contextual or confirmation bias may be introduced downstream in the investigative process
- e.g. gross example is ‘item X was recovered from suspect Y, a known repeat offender’
53
Q
What does undertaking forensic examinations at scenes require?
Why might there be scope for bias to affect these decisions?
A
- requires an understanding of the investigative needs of the case
- to observe, discover and recover evidence to meet both present needs and those anticipated for the future
- if guidance for these decision-making processes is not explicitly documented
- then actions taken at this stage are largely reliant on examiners intuition and tacit knowledge which in turn are susceptible to bias
54
Q
Why might there be scope for bias to affect communication of findings to others through written reports rather than verbal updates
A
- whilst slower, it is preferable as the writing provides less risk of introducing bias into the transfer of information
- written communication is more transparent about the decision making process
55
Q
DNA mixtures
A
- items are received along with case information and questions to be addressed by scientific work
- the case information supplied by law enforcement customer is used to direct the DNA recovery and analysis strategy, ideally within a framework of appropriate propositions
- if non-complex DNA results are obtained that match a suspect, an appropriate random match probability or likelihood ratio (LR) estimate is assigned
- if complex mixed DNA results are obtained that can be numerically evaluated the probability of the mixed result is calculated under appropriate prosecution and defence hypotheses and a LR is assigned
- if complex DNA results are obtained that do not lend themselves to statistical evaluation, in some circumstances, a qualitative assessment is made and an opinion about the significance of the DNA results can be put forward
- findings are checked by a competent colleague/peer
- a statement or report is issued
- the scientist may be called to court to give oral testimony
56
Q
R v Pickering
A
- in 2012, Pickering was convicted at the Crown Court on one count of sexual assault on a girl under the age of 13
- he was sentenced to 30 months imprisonment and a sexual offences prevention order made
- Pickering’s partner had an arrangement that they would babysit for each other
- she left her children with Pickering and his partner for them to babysit with their own child
- she went to bed and he stayed up with them and had consumed lot of beer
- P’s evidence is that Pickering touched her vagina over her clothes
- he lifted up her top and touched and sucked her breasts and gave her a love bite
- he then licked/sucked her vagina
- she denied that Pickering had given her a wedgie and denied making allegation up
- next morning victim showed her father mark on her neck and told her parents what Pickering had done
- Pickering’s evidence was that he had tickled his son C and P and given them both a wedgie and he denied touching P appropriately
- knickers:
- amylase found in high concentration in gusset of knickers (this is main constituents of saliva where it is usually present in high, though variable concentrations although it is present in other body fluids, a high concentration is indicative of the fluid being saliva)
- mixed DNA profile contributed by at least three people
- all components of DNA profile of P and all but one component of Pickering were present
- there were two components definitely present and three other components possible present that could not have come from P pr Pickering
- left breast:
- couldn’t determine if saliva was present
- DNA analysis showed possible presence of DNA from more than one person
- all of the confirmed components corresponded with the components in Pickering’s DNA profile
- the other components found matched the components of P’s DNA profile
- vulva:
- not possible to determine if saliva was present
- DNA analysis showed the presence of DNA from more than one person
All of the components could have originated from either ‘P’ or Pickering - the vest:
- not possible to determine if saliva was present
- DNA analysis indicates the presence of DNA from at least three people
- all of the corresponding components from ‘P’ and Pickering were present
- there were components which could not have originated from ‘P’ or Pickering
- prosecution scientist:
- none of the DNA results were suitable for statistical analysis as there must have been at least one other unknown person who contributed to the mixed profile
- based on the scenario that Pickering had licked P’s left breast and vagina the findings could entirely be explained by her account.
- on Pickering’s account findings were unlikely
- he concluded that the results were more likely to be consistent with P’s account than Pickering’s account but he could not express a view on the strength of support is no statistical evaluation could be carried out
- The DNA evidence relating to the knickers should not be admitted in terms of either amylase or DNA
- appeal conclusions on admissibility of DNA evidence:
- if all of DNA evidence did was to show that the findings were consistent with ‘P’s’ account and that Pickering could not be excluded then its relevance was peripheral
- it was however prejudicial and confusing and should not have been admitted
- the case is solely on whether or not Pickering was a possible contributor and could not be excluded thus the contention that there was no statistical evidence or no evaluative evidence
- overall conclusion
- presence of amylase in knickers was strong supportive evidence of Ps account
- that account was also supported by evidence of recent complaint
- evidence was relevant and it was properly admitted and correctly summed up to the jury
- the conviction is safe
57
Q
Advancing technology of DNA
A
- in addition to the routine application of enhanced sensitivity techniques, multiplexes in 2015 frequently achieve results from low quantities of DNA
- the incidence of complex mixtures and of low template profiles exhibiting stochastic effects is increasing and so the conditions in which subjective opinion tends to be relied on are more commonly encountered
- as a consequence, there is an increasing risk of cognitive contamination affecting DNA evidence
58
Q
Dlugosz???
A
59
Q
Rules of disclosure?
A
- within england and wales no prior disclosure of defendants account is required
- this often means that DNA scientist is required to make their own suppositions about appropriate defence hypotheses - proxy proposition
60
Q
What are 2 mitigation strategies against bias in DNA currently deployed in UK and overseas?
A
1 - prior interpretation of the case sample result before the reference result is revealed
2 - formally noting the following from the DNA result, prior to comparison with the reference profile
- suitability to include or exclude
- assessment of the number of contributors
- level of representation of contributors
- potential for stochastic effects
- identification of likely/unlikely genotype combinations that might explain the mixture
61
Q
What is recommended for DNA profile interpretation in all circumstances?
A
- full checking
- via repeat interpretation by an experienced and competent colleague including prior interpretation of the case sample result before the reference result is known
- the check should be conducted
Independent of and uninfluenced - the reporting scientist, and should use original unmodified hard copy or electronic results that are free from annotation
62
Q
Why must case stains/samples for testing be selected carefully?
A
- to minimise the occurrence of mixtures and low template issues
- selection should be informed by case information
- duplicate (or multiple) analyses to assess stochastic effects in low template samples
- replication should be applied whenever a poor quality profile is to be relied on to progress an investigation or provide evidence against a suspect (improves the scientist’s ability to gauge accurately whether or not the sample is suitable for any form of comparison or statistical evaluation)
63
Q
How is analysis and interpretation of DNA mixtures carried out:
- cons
- pros
A
- carried out blind in complete absence of any info about case
- this approach is practised in some jurisdictions and eliminates the risk of some types of bias
- however it does present the practical challenge of separating
- case strategy
- hypotheses testing
- stain selection, etc.
- from result interpretation and reporting in the context of the case
- however it provides assurance for the courts that the interpretation is free from contextual bias
64
Q
When are interpretation software used?
A
- used for complex mixtures such as
- STRmix (Institute of Environmental Science and Research)
- true Allele® (Cybergenetics)
- ideally these methods should be used whenever other objective numerical methods are not appropriate
65
Q
What is good practice when a numerical evaluation is not possible?
A
- it remains of crucial importance that qualitative & subjective judgements of pertinent profile features and their combined likelihood are assessed under the hypotheses framed by both the prosecution hypothesis and defence hypothesis separately
- the final opinion of evidential weight must be based on how much, if any, comparison of separate assessments favours one hypothesis over the other, as with a likelihood ratio (LR)
66
Q
What must happen if there is no suitable option for objective evaluation?
A
- only employ qualitative and subjective-based approaches that have been validated and therefore have demonstrated robustness of resultant conclusions and opinions
- such procedures should include system performance data indicating when the approach breaks down and is no longer valid
67
Q
Summary of fingerprint enquiry report?
A
- improved note taking
- complex marks process to manage variance in opinion between examiners
- those involved in technical review process have no prior knowledge or access to case-related info or technical findings of any other examiners
- blind verification process for lead identifications in which verifying examiners have no knowledge of the technical findings of any previous examiners
- removal of any case context information or related communication documentation from the verification process in any circumstances
- regular dip-sampling of all completed casework
- training programmes for examiners exploring cognitive bias and its impact on the human decision-making process
68
Q
ACE-V
A
- most commonly accepted approach to fingerprint comparison in the UK and the USA
- process:
- examiner analyses a mark
- examiner then compares mark with known print
- having compared images, examiner evaluates what they have seen and reaches a decision
- results are then subject to verification by one or more additional examiners
69
Q
What are categories of cognitive bias in FP examination process
A
- contextual
- confirmation
- cultural
70
Q
contextual bias in FP examination process
A
- FP examiners are exposed to a wealth of contextual information, which will impact on their decision-making process
- nature and details of the crime including background information
- association with or personal knowledge of the victim or their circumstances
- status of suspects or person(s) already in custody for the crime
- previous criminal activity of suspects or persons of interest
- location of the crime (an area close to their home)
- media or public interest associated with the crime
- personal moral codes or behaviours
- time pressure from investigating officers or office managers
71
Q
confirmation bias in FP examination process
A
- police Officers or prosecution services, with both hoping that the examination outcomes will help to ‘solve the case’ or ‘secure a conviction’.
- contributing to the detection of crime is considered a fundamental aspect of fingerprint bureau service delivery
- personal identification or ‘hit’ rates are used as key performance indicators at both organisational and individual level
- having prior knowledge of the previous examiner’s findings and conclusions may also expose fingerprint examiners to the risk of confirmation bias
- having found a number of features from an unknown impression to agree with features in an impression from a known source, the examiner will then begin to reason backward, finding features in the unknown impression that are suggested by those in the known print
72
Q
cultural bias in FP examination process
A
- cultural influence that can impact on decision-making process include:
- strict hierarchical structures based on time served rather than competence
- over confidence in individual or organisational competence
- lack of interaction with peers or exposure to alternative methods of working
- lack of acceptance of the potential for errors or effective root cause analysis of errors
73
Q
Two examples of cases where cognitive bias was an issue
A
- brandon mayfield 2006 and shirley mckie 1999
74
Q
What was introduced by FBI after Brandon Mayfield case?
A
- blind verification
- independent application of Analysis, Comparison, and Evaluation (ACE) by another qualified examiner who does not know the conclusions of the primary examiner
- they take place in the following cases
- those with a single mark conclusion
- circumstances where there are conflicts between examiners
- circumstances where there are conflicts on decisions of ‘value’ or ‘no value’
75
Q
What is risk treatment of FP?
A
- introduce control mechanisms for consideration with regard to fingerprint examination:
- survey and breakdown the extent of current contextual information available to examiners and assess the added value that each piece of information brings to the examination process
- remove or limit contextual information that adds no tangible value to the fingerprint examination process
- remove or limit contextual information made available to verifying or subsequent examiners
- introduce a blind verification process
- as part of a technical review process for complex marks or circumstances where examiners have a variance in opinion, introduce an appropriate and proportionate note-taking strategy, which requires examiners to provide written and visual accounts of their reasoning and findings
- develop bespoke training programmes to raise awareness of the cognitive issues
- as part of an established quality management system, instigate an effective review and monitoring process to provide assurance that the risk treatment and control measures continue to provide effective risk management
76
Q
What is the generic forensic process for interpretation and reporting of comparison of ‘marks’ cases?
A
- examine item/mark recovered from the crime scene
- use recovery and enhancement techniques as required
- generate/examine control item
- make test marks if required in appropriate manner
- undertake a comparison using appropriate methods and equipment
- interpret and evaluate findings
- verify the result
- describe findings in a statement or report
- the scientist may be called to court to give oral testimony
77
Q
What happens in mark comparison?
A
- seeks to establish if a mark has been made by the submitted exhibit or has been made by the same item
- it is based on the comparison of detail and is therefore observational
- the scientist is looking to determine if the detail present in the mark matches characteristic detail on the item or in a test mark or is significantly different
78
Q
What is class, sub-class, invidial?
A
- class - unintentional manufacturing marks present on a sub-set of items
- sub-class - random damage/wear and tool mark characteristics
- individual - any examination is dependent on visual quality and clarity of detail observed by examiner
79
Q
final mark comparison
A
- final assessor of the level of significance of any agreement between the marks is the human operator, there is no significant instrumental analysis
- in footwear mark comparisons, methods are normally side-by-side comparisons or overlay
- footwear expert assesses the level of agreement in terms of pattern, pattern configuration, mould/moulding detail, wear and damage
- the assessment is subjective although reference material and data can be used to support evaluation of findings
80
Q
low risk of bias in marks
- detail
- equipment
- approach/examiner
- scientist/examiner
A
- detail in mark is clear, well define and ambiguous
- optimum visualisation of detail in a mark using appropriate equipment/imaging and enhancement techniques
- methodical approach with defined standards built on principles that have been tested and validated
- possible confirmation bias may reduce as a consequence of comparison reviewer having less contextual infomation
- scientist/examiners are well trained, experience and continuously meet acceptable standards of competence
81
Q
high risk of bias in marks
- detail
- equipment
- approach/examiner
- scientist/examiner
A
- marks are confused and complex, of poor quality and the detail present is poorly defined
- poor or inappropriate equipment/imaging and enhancement techniques
- when the approach is unresearched and personal to the operator
- scientists/examiners are inexperienced, unmonitored and left to adopt their own approach
82
Q
What are mitigation strategies of bias within marks?
- CAI
- disclosure
- firearms specific
- blind
- validation testing
- standards
- training
- training and education
- quality
- blind trials
A
- CAI - comparison of expected, pre-assessed outcomes of appropriate hypotheses with actual results
- full disclosure of all data used in the evaluation
- in firearms cases, reviewer should clearly set out what is official guidance and what is statue ensuring that alternative classification hypotheses are addressed to counter confirmation bias
- use completely blind checker who repeats full interpretation but in the absence of any contextual information relating to the case
- an acceptable alternative is that the result will be subject to a critical findings check by a second authorised examiner
- checker then undertakes a detailed independent review wherever possible without knowledge of the previous practitioners conclusion
- validation testing of qualitative and subjective-based approaches to demonstrate robustness of conclusions and opinions
- development of standards and quality managed procedures for qualitative and subjective-based methods, including system performance data indicating when the approach breaks down and is no longer valid
- practitioner training in the specific method used together with initial and ongoing competency assessment
- training and education in relation to risks of cognitive bias in firearms classification and marks’ comparisons
- an approach to quality that includes assessment and ongoing monitoring of competence of practitioners including the use of proficiency tests, declared and undeclared
- use of blind trials should be introduced to increase success rate of cold hits
83
Q
particulate trace evidence recovery and analysis
A
- glass, paint, hairs and fibres
- examiner is often presented with large amount of debris that potentially contains some of target material and other materials that have no relevance to the case
- comparison of crime (unknown-recovered) material with one or more known/reference samples using the appropriate microscopy and instrument/analytical techniques
- scientific findings are evaluated and interpreted within the context of the case-specific information available (source or activity level)
- report or statement describing findings and providing an opinion on their significance is then provided
84
Q
low risk of bias in marks
- case assessment
- examination process
- result quality
- interpretation approach
- operator competence
- checking
A
- full case assessment considering potential outcomes, preferably with at least two competing hypotheses
- empirical analysis using instrumental techniques
- results are clear and ambiguous
- there is a methodical approach with defined standards built on principles that have been tested and validated
- operators are well trained, experienced and meet acceptable standards of competence
- independent confirmation of critical observations
- full independent reinterpretation
85
Q
high risk of bias in marks
- case assessment
- examination process
- result quality
- interpretation approach
- operator competence
- checking
A
- no case assessment
- only one hypotheses
- subjective microscopic analysis only
- results show wide intra-sample variation, are of poor quality and there is an increased reliance on subjective opinion
- approach is unresearched and personal to operator
- operators are inexperienced, unmonitored and left to adopt their own approach
- no checking or checking is conducted collaboratively
86
Q
Case example where cognitive bias may contribute to error
A
- absence of matching glass fragments concluded as being inconclusive
- clothing submitted from suspect believed to have seen breaking a glass window
- high expectation of finding glass fragments on clothes
- no glass found - this provides no evaluation of evidence
- concluded that findings are inconclusive as it is not possible to comment as no glass was found
87
Q
Absence of matching fibres concluded as being neutral? problem with this
A
- examination of car seat tapings for transfer of fibres from clothing of individual who is alleged to have stolen and driven the car for some hours results in no matching fibres being found
- defendant made no comment
- tempting to conclude that absence of matching fibres is neutral and does not assist in addressing whether or not individual had been in car
- however
- if info available provides no explanation for absence of matching fibres (e.g. changing clothes before arrest) and scientist had high expectation of finding matching fibres if the contact had occurred as alleged
- then the absence of matching fibres may well support view that defendant had not been in car
88
Q
What is problem in the treatment of crime/reference material post transfer?
A
- fibre examiner faces difficulty if clothing has been altered at chemical level in period between offence and seizure of clothing e.g. body of victim been submerged in river or at sea for some time causing dye in clothing to fade
- challenge: searching for fibres without a reference sample that is representative of fabric at time of offence
89
Q
mitigation strategies for trace
A
- independent checking where only subjective observational assessments of a match are possible
- independent checking of analytical results where instrumental techniques are used
- use of statistical approach to evaluation (statistical approach can be applied rather than relying on experience of practitioner
- CAI - a robust and documented comparison of expected, pre-assessed outcomes with actual results under appropriate competing hypotheses
- training - appropriate training of practitioners in methods employed can demonstrate initial and ongoing competence
- quality assurance trials
90
Q
video and audio examination process
A
- recovery of video, photo or audio material related to the crime scene
- items are received by the analyst along with relevant case information and the questions to be addressed by the scientific work
- generation of an exact copy of the original, then use of techniques as required to clarify or clean up the copy of the image or audio signal
- examination of the copied material recovered from the crime scene and notation of features determined to be reliable
- examination of the ‘control’ item
- undertaking a comparison using appropriate methods and equipment
- interpreting and evaluating findings
- verification of the result
- findings are described in a statement or report
- the scientist may be called to court to give oral testimony
91
Q
low risk of bias in video
- detail and presentation
- equipment
- approach
- scientist/examiner
- verification of results
A
- the images are clear detailed and unambiguous with item and reference images generated under identical conditions
- optimum visualisation of the detail in an image using appropriate equipment/imaging and enhancement techniques
- there is methodical approach with defined standards built on principles that have been tested and validated
- item is characterized prior to exposure to reference images
- scientist/examiners are well trained, experienced and continuously meet acceptable standards of competence
- independent review of critical findings
92
Q
high risk of bias in video
- detail and presentation
- equipment
- approach
- scientist/examiner
- verification of results
A
- images are of poor quality and the detail present is poorly defined and the images being compared have been generated under very different conditions
- poor or inappropriate equipment/imaging and enhancement techniques
- when the approach is unresearched and personal to operator
- item is characterised after exposure to reference image
- scientist/examiners are inexperienced, unmonitored and left to adopt their own approach
- there is no independent review or reviewer knows findings and conclusions drawn from original assessment
93
Q
Staff training and awareness of cognitive issues sit within?
A
- quality management system
94
Q
there are several types of contextual information of concern, what are these?
A
- five level taxonomy
1 - trace evidence
2 - reference materials
3 - the case information
4 - base rate expectations that arise from their experience e.g. when the examiner expects a particular result
5 - organisational and cultural factors
95
Q
case managers
A
- a straightforward tool for dealing with bias from case information (level 3)
- designed to prevent contextual bias by protecting the examiner from exposure to task-irrelevant information
96
Q
Bias vs task relevance
A
- some types of information while potentially biasing may also be task relevant e.g. evidence that analysts must examine to perform their duties may contain information that is potentially biasing
- for example, cases in which Level 1 information, the trace evidence being evaluated, contains contextual information (e.g., blood spatter patterns that contain information about the nature of the crime, or handwriting and voice samples in which the meaning of the words is potentially biasing)
97
Q
Why can reference samples be biasing
A
- these samples are clearly relevant because analyst must compare them to trace evidence samples to determine whether they are similar enough to conclude that they come from same source
- but it is possible that an analysts interpretation of the trace evidence might inadvertently be influenced by knowing the characteristics of the reference samples
98
Q
linear sequential unmasking
A
- this approach is linear in the sense that one must begin with the trace evidence before being exposed to and working with reference material, thus working from the evidence to the suspect, rather than from the suspect to the evidence
- this approach does not prevent exposure to biasing relevant information
- it does mandate that this information be presented as late as possible in the examination process and only when it is necessary
- a critical element of sequential unmasking is that forensic examiner must first examine and document the trace evidence from the crime scene (level 1) before being exposed to the known reference material (level 2)
- according to sequential unmasking examiners are permitted to revisit, as well as change, their initial analysis of the trace evidence once they have reviewed the reference material, provided that they document these changes
99
Q
What are the 7 factors affecting decision making
A
- case evidence
- reference materials
- irrelevant case information
- base rate expectations
- organisational factors
- training and motivation
- cognitive architecture and brain
100
Q
What are ways to mitigate bias in forensic investigation?
A
- removing irrelevant contextual information from samples prior to running tests (case management)
- minimising bias in expect outcomes (blind verification)
