Bias Flashcards

Question

survivorship bias

Answer 1

- an error that comes from focusing only on surviving examples causing us to misjudge a situation

Answer 2

- that an experts duty to the court includes the following: - an expert must help the court to achieve the overriding objective by giving an opinion that is objective and unbiased and within the expert's area or areas of expertise - this duty overrides any obligation to the person from whom the expert receives instructions or by whom the expert is paid - used to be part 33

Answer 3

- to evaluate scientific findings and the results of analytical tests in the context of the relevant case circumstances

Answer 4

- balanced - robust - logical - transparent

Answer 5

- when results are clear and unambiguous - a methodical approach - with defined standards - built on principles that have been tested and validated - when practitioners and checkers are well trained, experienced and continuously meet acceptable standards of competence - when interpretation is checked by a competent peer who conducts a separate interpretation fully independent and without influence from the reporting scientist

Answer 6

- when results are: - complex - of poor quality - and there is an increased reliance on subjective opinion - when the approach is unresearched and personal to practitioner - when practitioners and checkers are inexperienced, unmonitored and left to adopt their own approach - when checking is less rigorous and/or conducted collaboratively

Answer 7

- reduction or elimination of impact of bias in decision making and problem solving

Answer 8

- practice of obtaining reliable information about physical objects through the processes of recording measuring and interpreting photographic images

Answer 9

- exposure to other information that is irrelevant to the assessment but that introduces subconscious bias into findings

Answer 10

1 - being the sole reviewer of their critical findings 2 - being over-familiar with or trusting another person instead of relying on objective evidence 3 - having organisational and management structures that could be perceived to reward, encourage or support bias (where for example a culture of performance measurement and time pressures could potentially pressurise examiners into biasing decisions)

Answer 11

- often vital for constructing and testing relevant hypotheses and propositions - however if knowing about certain aspects are assessed to work against the objective process in a particular method - if whole case file is handed over to an analyst with all extraneous detail then even if there is no perceptible bias there is the perception that it could have occurred and may be open to challenge in court

Answer 12

- varies between individuals

Answer 13

- through a proficiency-testing programme - utilising mocked up casework samples for which the expected outcomes of testing and evaluation are known - blind trials are gold standard in providing most reliable indicator of real-life performance but in reality they can be very time-consuming - good practice is to undertake a mixed programme of both declared and undeclared trials, with the proficiency of all individuals tested on a regular basis

Answer 14

- practitioners need training in how the evaluation procedures assist in overcoming some of these risks in their respective roles, as well as what residual risk may remain - ideally this training should be practical - given that susceptibility to psychological and cognitive influences varies between individuals, there may be merit in assessing these susceptibilities as part of the recruitment or selection procedures for new staff

Answer 15

- by taking contemp notes or technical records - accordance with this requirement wherever it is practicable to do so at all stages in the collection and processing of forensic evidence provides the best safeguard against potential reconstructive effects

Answer 16

- where scientists are subconsciously influenced by acting on behalf of the defence or prosecution - difficult to demonstrably eliminate given the adversarial nature of the CJS within the UK - potentially compounded by the pressures of a commercial market in which a supplier/customer relationship - these pressures apply whether an FSP is providing contracted services to the prosecution or defence or in the case of police laboratories in providing services to an internal customer

Answer 17

1 - define requirement 2 - develop an examination strategy 3 - agree examination strategy with the client 4 - lead scientist instructs analyst to carry out forensic examinations and analyses 5 - review quality and content of examination results 6 - compare results with reference samples and marks 7 - evaluate and interpret the scientific findings and analytical tests 8 - second expert verifies the findings 9 - communicate scientific findings and analytical tests

Answer 18

- to record, retain and reveal their work (disclose) - including the following - record all info received and record details of interpretation

Answer 19

- a formal process of pre-assessing the expected probabilities for a realistic range of possible outcome in as many or as few categories as is sensible for the examination - recording their opinions - may be that possible outcomes could be simply the two hypotheses that the items being considered either match or do not match - it may be that each category in the realistic list of outcomes is considered firstly under the assumption that the prosecution hypothesis is true, and secondly under the assumption that the defence hypothesis is true - these are used to provide an expected outcome that may be either qualitative or quantitative (LR)

Answer 20

- identify whether the scientist’s role in the case is investigative (for example, intelligence) or evaluative (judicial) - seek clarity on what are the issues, the purpose and how this fits into the hierarchy of sub-source (for example, touch DNA), source, activity and offence level propositions

Answer 21

- formulate relevant prosecution and defence alternatives based on the case circumstances and information provided - consider any agreed assumptions that are used in formulating these alternatives - use assessment of possible outcomes to determine which tests are most informative and discriminating - use this pre-assessment to assign a weight to an exhaustive list of possible outcomes, giving the expected outcome for each, expressed as a LR where these are quantitative - this approach provides clarity on the alternatives being considered, and the pre-assessment of weight for all outcomes avoids the potential bias of using the observed results to assign weight of evidence

Answer 22

- review the quality and content of examination results - decisions on suitability of the results and marks for later comparison are made at this stage, to avoid post-comparison rationalisation of opinion on quality - compare the results with the reference samples and marks - quality and suitability of the questioned result has already been assessed so this is not influenced by the reference result

Answer 23

- mitigate the confirmation bias by using the LR or qualitative expectation that was assigned to each outcome before the examinations and tests were performed - pre-assessment enables the scientist to explain how the weight of the evidence has been assigned - provide details of the assumptions that have been made - give the basis of the expert opinion and specify the propositions considered, with the reasoning for these, based on the case context - include any limitation of the opinion - where there is a range of opinion on the matters dealt with in the report, the expert’s report must summarise the range of opinion and give reasons for the expert’s own opinion as required by the Criminal Procedure Rules - verification by a second expert (independent review at this stage in advance of communicating the result to the client) - communicate the scientific findings and analytical tests

Answer 24

- preserving scene - securing evidence - speed of response - proportionate use of resources based on seriousness of crime - all are overridden by most pressing priority - preservation of life

Answer 25

- could result in failure to secure required evidence if a crime scene investigation is closed prematurely resulting in crucial evidence being lost - could mislead an investigation by investigators focusing too early and incorrectly on a false lead and other evidence is potentially overlooked - if undertaken incorrectly activities could result in psychological contamination of evidence downstream in forensic analysis and interpretation process

Answer 26

- gathering information prior to scene attendance - prior to scene attendance information is gathered from any available source regarding the incident to be investigated - this may include witness or victim accounts as to what is alleged to have happened and by their nature these may be consciously or subconsciously biased

Answer 27

- determining locations and boundaries of the scene - the entry/exit points of the offender based on observations - information received and inferences - past experiences of an individual on which they may base their decisions are subjective - other factors such as convenience may be more relevant and have more impact in real life e.g. establishing boundary by taping from lamp post to lamp post as they are already there

Answer 28

- selection of equipment - decisions on which images to capture - entails an assessment of the current case needs plus some anticipation of future needs - depending on how the written record is crafted, there is a risk that contextual or confirmation bias may be introduced downstream in the investigative process - e.g. gross example is ‘item X was recovered from suspect Y, a known repeat offender’

Answer 29

- requires an understanding of the investigative needs of the case - to observe, discover and recover evidence to meet both present needs and those anticipated for the future - if guidance for these decision-making processes is not explicitly documented - then actions taken at this stage are largely reliant on examiners intuition and tacit knowledge which in turn are susceptible to bias

Answer 30

- whilst slower, it is preferable as the writing provides less risk of introducing bias into the transfer of information - written communication is more transparent about the decision making process

Answer 31

- items are received along with case information and questions to be addressed by scientific work - the case information supplied by law enforcement customer is used to direct the DNA recovery and analysis strategy, ideally within a framework of appropriate propositions - if non-complex DNA results are obtained that match a suspect, an appropriate random match probability or likelihood ratio (LR) estimate is assigned - if complex mixed DNA results are obtained that can be numerically evaluated the probability of the mixed result is calculated under appropriate prosecution and defence hypotheses and a LR is assigned - if complex DNA results are obtained that do not lend themselves to statistical evaluation, in some circumstances, a qualitative assessment is made and an opinion about the significance of the DNA results can be put forward - findings are checked by a competent colleague/peer - a statement or report is issued - the scientist may be called to court to give oral testimony

Answer 32

- in 2012, Pickering was convicted at the Crown Court on one count of sexual assault on a girl under the age of 13 - he was sentenced to 30 months imprisonment and a sexual offences prevention order made - Pickering's partner had an arrangement that they would babysit for each other - she left her children with Pickering and his partner for them to babysit with their own child - she went to bed and he stayed up with them and had consumed lot of beer - P's evidence is that Pickering touched her vagina over her clothes - he lifted up her top and touched and sucked her breasts and gave her a love bite - he then licked/sucked her vagina - she denied that Pickering had given her a wedgie and denied making allegation up - next morning victim showed her father mark on her neck and told her parents what Pickering had done - Pickering's evidence was that he had tickled his son C and P and given them both a wedgie and he denied touching P appropriately - knickers: - amylase found in high concentration in gusset of knickers (this is main constituents of saliva where it is usually present in high, though variable concentrations although it is present in other body fluids, a high concentration is indicative of the fluid being saliva) - mixed DNA profile contributed by at least three people - all components of DNA profile of P and all but one component of Pickering were present - there were two components definitely present and three other components possible present that could not have come from P pr Pickering - left breast: - couldn't determine if saliva was present - DNA analysis showed possible presence of DNA from more than one person - all of the confirmed components corresponded with the components in Pickering's DNA profile - the other components found matched the components of P's DNA profile - vulva: - not possible to determine if saliva was present - DNA analysis showed the presence of DNA from more than one person All of the components could have originated from either ‘P’ or Pickering - the vest: - not possible to determine if saliva was present - DNA analysis indicates the presence of DNA from at least three people - all of the corresponding components from ‘P’ and Pickering were present - there were components which could not have originated from ‘P’ or Pickering - prosecution scientist: - none of the DNA results were suitable for statistical analysis as there must have been at least one other unknown person who contributed to the mixed profile - based on the scenario that Pickering had licked P’s left breast and vagina the findings could entirely be explained by her account. - on Pickering's account findings were unlikely - he concluded that the results were more likely to be consistent with P’s account than Pickering's account but he could not express a view on the strength of support is no statistical evaluation could be carried out - The DNA evidence relating to the knickers should not be admitted in terms of either amylase or DNA - appeal conclusions on admissibility of DNA evidence: - if all of DNA evidence did was to show that the findings were consistent with ‘P’s’ account and that Pickering could not be excluded then its relevance was peripheral - it was however prejudicial and confusing and should not have been admitted - the case is solely on whether or not Pickering was a possible contributor and could not be excluded thus the contention that there was no statistical evidence or no evaluative evidence - overall conclusion - presence of amylase in knickers was strong supportive evidence of Ps account - that account was also supported by evidence of recent complaint - evidence was relevant and it was properly admitted and correctly summed up to the jury - the conviction is safe

Answer 33

- in addition to the routine application of enhanced sensitivity techniques, multiplexes in 2015 frequently achieve results from low quantities of DNA - the incidence of complex mixtures and of low template profiles exhibiting stochastic effects is increasing and so the conditions in which subjective opinion tends to be relied on are more commonly encountered - as a consequence, there is an increasing risk of cognitive contamination affecting DNA evidence

Answer 34

- within england and wales no prior disclosure of defendants account is required - this often means that DNA scientist is required to make their own suppositions about appropriate defence hypotheses - proxy proposition

Answer 35

1 - prior interpretation of the case sample result before the reference result is revealed 2 - formally noting the following from the DNA result, prior to comparison with the reference profile - suitability to include or exclude - assessment of the number of contributors - level of representation of contributors - potential for stochastic effects - identification of likely/unlikely genotype combinations that might explain the mixture

Answer 36

- full checking - via repeat interpretation by an experienced and competent colleague including prior interpretation of the case sample result before the reference result is known - the check should be conducted Independent of and uninfluenced - the reporting scientist, and should use original unmodified hard copy or electronic results that are free from annotation

Answer 37

- to minimise the occurrence of mixtures and low template issues - selection should be informed by case information - duplicate (or multiple) analyses to assess stochastic effects in low template samples - replication should be applied whenever a poor quality profile is to be relied on to progress an investigation or provide evidence against a suspect (improves the scientist’s ability to gauge accurately whether or not the sample is suitable for any form of comparison or statistical evaluation)

Answer 38

- carried out blind in complete absence of any info about case - this approach is practised in some jurisdictions and eliminates the risk of some types of bias - however it does present the practical challenge of separating - case strategy - hypotheses testing - stain selection, etc. - from result interpretation and reporting in the context of the case - however it provides assurance for the courts that the interpretation is free from contextual bias

Answer 39

- used for complex mixtures such as - STRmix (Institute of Environmental Science and Research) - true Allele® (Cybergenetics) - ideally these methods should be used whenever other objective numerical methods are not appropriate

Answer 40

- it remains of crucial importance that qualitative & subjective judgements of pertinent profile features and their combined likelihood are assessed under the hypotheses framed by both the prosecution hypothesis and defence hypothesis separately - the final opinion of evidential weight must be based on how much, if any, comparison of separate assessments favours one hypothesis over the other, as with a likelihood ratio (LR)

Answer 41

- only employ qualitative and subjective-based approaches that have been validated and therefore have demonstrated robustness of resultant conclusions and opinions - such procedures should include system performance data indicating when the approach breaks down and is no longer valid

Answer 42

- improved note taking - complex marks process to manage variance in opinion between examiners - those involved in technical review process have no prior knowledge or access to case-related info or technical findings of any other examiners - blind verification process for lead identifications in which verifying examiners have no knowledge of the technical findings of any previous examiners - removal of any case context information or related communication documentation from the verification process in any circumstances - regular dip-sampling of all completed casework - training programmes for examiners exploring cognitive bias and its impact on the human decision-making process

Answer 43

- most commonly accepted approach to fingerprint comparison in the UK and the USA - process: - examiner analyses a mark - examiner then compares mark with known print - having compared images, examiner evaluates what they have seen and reaches a decision - results are then subject to verification by one or more additional examiners

Answer 44

- contextual - confirmation - cultural

Answer 45

- FP examiners are exposed to a wealth of contextual information, which will impact on their decision-making process - nature and details of the crime including background information - association with or personal knowledge of the victim or their circumstances - status of suspects or person(s) already in custody for the crime - previous criminal activity of suspects or persons of interest - location of the crime (an area close to their home) - media or public interest associated with the crime - personal moral codes or behaviours - time pressure from investigating officers or office managers

Answer 46

- police Officers or prosecution services, with both hoping that the examination outcomes will help to ‘solve the case’ or ‘secure a conviction’. - contributing to the detection of crime is considered a fundamental aspect of fingerprint bureau service delivery - personal identification or ‘hit’ rates are used as key performance indicators at both organisational and individual level - having prior knowledge of the previous examiner’s findings and conclusions may also expose fingerprint examiners to the risk of confirmation bias - having found a number of features from an unknown impression to agree with features in an impression from a known source, the examiner will then begin to reason backward, finding features in the unknown impression that are suggested by those in the known print

Answer 47

- cultural influence that can impact on decision-making process include: - strict hierarchical structures based on time served rather than competence - over confidence in individual or organisational competence - lack of interaction with peers or exposure to alternative methods of working - lack of acceptance of the potential for errors or effective root cause analysis of errors

Answer 48

- brandon mayfield 2006 and shirley mckie 1999

Answer 49

- blind verification - independent application of Analysis, Comparison, and Evaluation (ACE) by another qualified examiner who does not know the conclusions of the primary examiner - they take place in the following cases - those with a single mark conclusion - circumstances where there are conflicts between examiners - circumstances where there are conflicts on decisions of ‘value’ or ‘no value’

Answer 50

- introduce control mechanisms for consideration with regard to fingerprint examination: - survey and breakdown the extent of current contextual information available to examiners and assess the added value that each piece of information brings to the examination process - remove or limit contextual information that adds no tangible value to the fingerprint examination process - remove or limit contextual information made available to verifying or subsequent examiners - introduce a blind verification process - as part of a technical review process for complex marks or circumstances where examiners have a variance in opinion, introduce an appropriate and proportionate note-taking strategy, which requires examiners to provide written and visual accounts of their reasoning and findings - develop bespoke training programmes to raise awareness of the cognitive issues - as part of an established quality management system, instigate an effective review and monitoring process to provide assurance that the risk treatment and control measures continue to provide effective risk management

Answer 51

- examine item/mark recovered from the crime scene - use recovery and enhancement techniques as required - generate/examine control item - make test marks if required in appropriate manner - undertake a comparison using appropriate methods and equipment - interpret and evaluate findings - verify the result - describe findings in a statement or report - the scientist may be called to court to give oral testimony

Answer 52

- seeks to establish if a mark has been made by the submitted exhibit or has been made by the same item - it is based on the comparison of detail and is therefore observational - the scientist is looking to determine if the detail present in the mark matches characteristic detail on the item or in a test mark or is significantly different

Answer 53

- class - unintentional manufacturing marks present on a sub-set of items - sub-class - random damage/wear and tool mark characteristics - individual - any examination is dependent on visual quality and clarity of detail observed by examiner

Answer 54

- final assessor of the level of significance of any agreement between the marks is the human operator, there is no significant instrumental analysis - in footwear mark comparisons, methods are normally side-by-side comparisons or overlay - footwear expert assesses the level of agreement in terms of pattern, pattern configuration, mould/moulding detail, wear and damage - the assessment is subjective although reference material and data can be used to support evaluation of findings

Answer 55

- detail in mark is clear, well define and ambiguous - optimum visualisation of detail in a mark using appropriate equipment/imaging and enhancement techniques - methodical approach with defined standards built on principles that have been tested and validated - possible confirmation bias may reduce as a consequence of comparison reviewer having less contextual infomation - scientist/examiners are well trained, experience and continuously meet acceptable standards of competence

Answer 56

- marks are confused and complex, of poor quality and the detail present is poorly defined - poor or inappropriate equipment/imaging and enhancement techniques - when the approach is unresearched and personal to the operator - scientists/examiners are inexperienced, unmonitored and left to adopt their own approach

Answer 57

- CAI - comparison of expected, pre-assessed outcomes of appropriate hypotheses with actual results - full disclosure of all data used in the evaluation - in firearms cases, reviewer should clearly set out what is official guidance and what is statue ensuring that alternative classification hypotheses are addressed to counter confirmation bias - use completely blind checker who repeats full interpretation but in the absence of any contextual information relating to the case - an acceptable alternative is that the result will be subject to a critical findings check by a second authorised examiner - checker then undertakes a detailed independent review wherever possible without knowledge of the previous practitioners conclusion - validation testing of qualitative and subjective-based approaches to demonstrate robustness of conclusions and opinions - development of standards and quality managed procedures for qualitative and subjective-based methods, including system performance data indicating when the approach breaks down and is no longer valid - practitioner training in the specific method used together with initial and ongoing competency assessment - training and education in relation to risks of cognitive bias in firearms classification and marks' comparisons - an approach to quality that includes assessment and ongoing monitoring of competence of practitioners including the use of proficiency tests, declared and undeclared - use of blind trials should be introduced to increase success rate of cold hits

Answer 58

- glass, paint, hairs and fibres - examiner is often presented with large amount of debris that potentially contains some of target material and other materials that have no relevance to the case - comparison of crime (unknown-recovered) material with one or more known/reference samples using the appropriate microscopy and instrument/analytical techniques - scientific findings are evaluated and interpreted within the context of the case-specific information available (source or activity level) - report or statement describing findings and providing an opinion on their significance is then provided

Answer 59

- full case assessment considering potential outcomes, preferably with at least two competing hypotheses - empirical analysis using instrumental techniques - results are clear and ambiguous - there is a methodical approach with defined standards built on principles that have been tested and validated - operators are well trained, experienced and meet acceptable standards of competence - independent confirmation of critical observations - full independent reinterpretation

Answer 60

- no case assessment - only one hypotheses - subjective microscopic analysis only - results show wide intra-sample variation, are of poor quality and there is an increased reliance on subjective opinion - approach is unresearched and personal to operator - operators are inexperienced, unmonitored and left to adopt their own approach - no checking or checking is conducted collaboratively

Answer 61

- absence of matching glass fragments concluded as being inconclusive - clothing submitted from suspect believed to have seen breaking a glass window - high expectation of finding glass fragments on clothes - no glass found - this provides no evaluation of evidence - concluded that findings are inconclusive as it is not possible to comment as no glass was found

Answer 62

- examination of car seat tapings for transfer of fibres from clothing of individual who is alleged to have stolen and driven the car for some hours results in no matching fibres being found - defendant made no comment - tempting to conclude that absence of matching fibres is neutral and does not assist in addressing whether or not individual had been in car - however - if info available provides no explanation for absence of matching fibres (e.g. changing clothes before arrest) and scientist had high expectation of finding matching fibres if the contact had occurred as alleged - then the absence of matching fibres may well support view that defendant had not been in car

Answer 63

- fibre examiner faces difficulty if clothing has been altered at chemical level in period between offence and seizure of clothing e.g. body of victim been submerged in river or at sea for some time causing dye in clothing to fade - challenge: searching for fibres without a reference sample that is representative of fabric at time of offence

Answer 64

- independent checking where only subjective observational assessments of a match are possible - independent checking of analytical results where instrumental techniques are used - use of statistical approach to evaluation (statistical approach can be applied rather than relying on experience of practitioner - CAI - a robust and documented comparison of expected, pre-assessed outcomes with actual results under appropriate competing hypotheses - training - appropriate training of practitioners in methods employed can demonstrate initial and ongoing competence - quality assurance trials

Answer 65

- recovery of video, photo or audio material related to the crime scene - items are received by the analyst along with relevant case information and the questions to be addressed by the scientific work - generation of an exact copy of the original, then use of techniques as required to clarify or clean up the copy of the image or audio signal - examination of the copied material recovered from the crime scene and notation of features determined to be reliable - examination of the ‘control’ item - undertaking a comparison using appropriate methods and equipment - interpreting and evaluating findings - verification of the result - findings are described in a statement or report - the scientist may be called to court to give oral testimony

Answer 66

- the images are clear detailed and unambiguous with item and reference images generated under identical conditions - optimum visualisation of the detail in an image using appropriate equipment/imaging and enhancement techniques - there is methodical approach with defined standards built on principles that have been tested and validated - item is characterized prior to exposure to reference images - scientist/examiners are well trained, experienced and continuously meet acceptable standards of competence - independent review of critical findings

Answer 67

- images are of poor quality and the detail present is poorly defined and the images being compared have been generated under very different conditions - poor or inappropriate equipment/imaging and enhancement techniques - when the approach is unresearched and personal to operator - item is characterised after exposure to reference image - scientist/examiners are inexperienced, unmonitored and left to adopt their own approach - there is no independent review or reviewer knows findings and conclusions drawn from original assessment

Answer 68

- quality management system

Answer 69

- five level taxonomy 1 - trace evidence 2 - reference materials 3 - the case information 4 - base rate expectations that arise from their experience e.g. when the examiner expects a particular result 5 - organisational and cultural factors

Answer 70

- a straightforward tool for dealing with bias from case information (level 3) - designed to prevent contextual bias by protecting the examiner from exposure to task-irrelevant information

Answer 71

- some types of information while potentially biasing may also be task relevant e.g. evidence that analysts must examine to perform their duties may contain information that is potentially biasing - for example, cases in which Level 1 information, the trace evidence being evaluated, contains contextual information (e.g., blood spatter patterns that contain information about the nature of the crime, or handwriting and voice samples in which the meaning of the words is potentially biasing)

Answer 72

- these samples are clearly relevant because analyst must compare them to trace evidence samples to determine whether they are similar enough to conclude that they come from same source - but it is possible that an analysts interpretation of the trace evidence might inadvertently be influenced by knowing the characteristics of the reference samples

Answer 73

- this approach is linear in the sense that one must begin with the trace evidence before being exposed to and working with reference material, thus working from the evidence to the suspect, rather than from the suspect to the evidence - this approach does not prevent exposure to biasing relevant information - it does mandate that this information be presented as late as possible in the examination process and only when it is necessary - a critical element of sequential unmasking is that forensic examiner must first examine and document the trace evidence from the crime scene (level 1) before being exposed to the known reference material (level 2) - according to sequential unmasking examiners are permitted to revisit, as well as change, their initial analysis of the trace evidence once they have reviewed the reference material, provided that they document these changes

Answer 74

- case evidence - reference materials - irrelevant case information - base rate expectations - organisational factors - training and motivation - cognitive architecture and brain

Answer 75

- removing irrelevant contextual information from samples prior to running tests (case management) - minimising bias in expect outcomes (blind verification)