L10 - Metacognition and Confidence Flashcards
What can we use in meta-cognition to measure the subjective ‘strength’ of a memory?
A persons confidence
In a controlled lab setting ‘Accuracy’ and ‘Confidence’ are two measures of the same underlying construct
What construct?
Memory Strength
Why is it important to determine whether the latent variable (confidence) is predictive of observed behaviour?
In real world settings we don’t know what the truth is, so we need to know if confidence is a predictor of accuracy.
Police are heavily influenced by eyewitness ______ as are jurors
Confidence
What is the name of the criteria used by the court that asks:
“How certain were you that the person you identified was the person you saw commit the crime?
Biggers’ criteria (U.S.)
In what circumstance can confidence be a good predictor of accuracy?
What is this called?
If confidence is measured immediately after the identification decision
and
there is no feedback from anyone else
Called: Pristine Confidence
What did Kelly and Lindsay (1993) discover relating the confidence?
Confidence can be changed by factors unrelated to memory strength
What is a point-bi-serial correlation?
Tells you the relationship between a categorical accuracy variable (e.g. 1 is correct 0 incorrect) and a continuous variable (e.g. subjective percentage confidence from 0-100%)
e.g. Spearman rank correlation or non-parametric correlation
87% of eyewitness researchers endorsed “confidence is not a good predictor of accuracy” in the past (Kassin et al. 2001)
Why?
Most researchers used a point-bi-serial correlation to analyse their data
The problem with this is it isn’t a good way of characterising the relationship.
Researchers were also using one set of stimulus materials for all witnesses
Wasn’t representative of stimulus materials and was not generalisable to the real world (a restriction of range of the correlation that you observed)
What are the 3 issues with using a point-bi-serial correlation to measure to confidence-accuracy relationship?
1) Cannot ‘see’ the full relationship between each level of confidence and accuracy
2) The correlation provides no info about the extent to which participants over/underestimate the probability that their decisions were correct
3) Point-bi-serial correlation can be low but calibration perfect
* e.g. you can get a low correlation (.23) but if you do a calibration plot their confidence tracks their accuracy really well*
What is a better way to measure the relationship between confidence and accuracy and how much insight people have into their memory?
Confidence-Accuracy Calibration Curve
Plot subjective confidence against proportion correct.
What is the calibration statistic that you obtain using Confidence-Accuracy Calibration?
What does it range from?
What does it tell you?
Calibration Stat: C Stat
0 = Perfect; 1 = Hopeless
Tells you how close the curve is to the perfect line of calibration.
It quantifies the distance between the actual calibration curve and the perfect calibration line.
What is the over/underconfidence statistic that you obtain using Confidence-Accuracy Calibration?
What does it range from?
What does it tell you?
Over/Underconfidence Statistic = O/U
Ranges from -1 to +1 (it is the mean confidence - (minus) mean proportion correct
Tells you how much more confident you are than accurate
What is the normalised resolution statistic that you obtain using Confidence-Accuracy Calibration?
What does it range from?
What does it tell you?
Statistic = NRI
Ranges from 0 to 1
Tells you how well confidence discriminates correct from incorrect responses
What are the 3 factors that affect calibration in confidence-accuracy relationships?
Decision Type
Reflection
Hypothesis Disconfirmation
What are the decision types that might affect confidence-accuracy calibration in eyewitness studies?
Postive decisions vs. Negative decisions
Positive decision: Where you make a choice (yes I have seen that person before)
Negative decision: No I haven’t seen that person before
What decision type is better calibrated for confidence-accuracy; Positive or Negative decisions?
Positive Decisions
Negative has barely any correlation to confidence
What findings did Brewer, Keast and Rishworth (2002) discover when asking participants to “reflect on their encoding conditions”?
It can improve the monitoring of memory
Shows much better calibration if you reflect on your encoding and scale your confidence accordingly
How does hypothesis disconformation (thinking about why you might be wrong) affect the confidence-accuracy relationship?
Makes them underconfident in their decisions
What was the hypothesis of the Busy et al. (2000) study?
If confidence is based only on memory strength then should see a single dimension emerge from as “state-trace analysis”
What IV’s did Busey et al. (2000) manipulate in their experiment?
Luminance and Rehearsal
Explain the predictions of the single-dimensional model and multi-dimensional model made in this model described in the Busey et al. (2000) study.
Single-Dimensional Model: Strength is a function of the combination of luminance and rehearsal.
This is experessed in an increase in accuracy and confidence
(Possible) Multi-Dimensional Model: Two underlying latent variables, rehearsal might affect both strength and certainty but luminance only affects strength.
If multi-dimensional, you will see a difference in the degree in which accuracy and confidence change as a function of those manipulations.
What does state-trace analysis allow you to assess?
Allows you to assess the dimensionality of a relationship
e.g. like a scatterplot
How did Busey et al. use “state-trace analysis” to determine whether confidence and accuracy relationship followed the single-dimensional model or the multidimensional model?
If two curves fall on top of one another = single dimension
If they fall in different locations = multiple dimensions
What were the results of the Busey et al. (2000) experiment 1 (rehearsal dimension)?
They manipulated stimulus duration.
In the rehearsal vs. no rehearsal condition it looked like there was a single dimensional plot.
Rehearsal is one-dimensional
In Busey et al. (2000) luminance experiment (experiment 3), what were the results?
Followed a multi-dimensional model (plot)
The extent that luminance is observed is more indicative of confidence than memory strength alone.
It boosts confidence more than just memory strength.
What were the conclusions of the Busey et al. (2000) for experiment 1+2?
Rehearsal experiment
What were the conclusions of the Busey et al. (2000) for experiment 3?
