Psychological model fitting

Question 1

Describe what a 2AFC task is

Answer 1

2 ALTERNATIVE FORCED CHOICE TASK
A task commonly used in the field of psychophysics to investigate the relationship between physical stimuli and mental phenomena.
• 2 possible trials: A (stimulus present) and B (stimulus absent)
• 2 possible responses: A (stimulus present) and B (stimulus absent)
• A widely used DV is number correct or percentage correct (hit + correct rejections, ignoring performance for individual stimuli)

Question 2

What are the 2 latent mechanisms in signal detection

Answer 2

1. Discrimination ability: ability to detect a stimulus
    An individual differences measure potentially similar to IQ
2. Bias: willingness/reluctance to respond one way or another
    Dependent on other factors

Using % correct as a DV conflates these 2 mechanisms

Question 3

What are the problems with using percentage correct as a DV in 2AFC

Answer 3

o % correct is hits + CR
But the presence and absence of a stimulus are not the same and thus conflating performance across the two may be problematic
o Detecting a stimulus might be easier than detecting its absence
o An individual might have an inherent bias towards one response.
o Stimulus detection task with 100 trials, 50 present, 50 absent
o PP1: randomly responds “present” 90% of the time will get 45 present trials correct and 5 absent trials correct
o PP2: randomly responds “absent” 90% of the time will get 45 absent trials correct and 5 present trials correct
o Both PPs get 50% accuracy
o We won’t know the biases because we conflate scores and don’t dissociate them

Question 4

Describe the problem of bias in psychophysics tasks such as 2AFC

Answer 4

•Bias: willingness/reluctance to respond one way or another
Measuring threshold of hearing: sound present or not
o If you start with sub-threshold sounds and progressively get louder, a person with tinnitus might respond “yes” to all/most of the trials and get a very high response score(high hit rate). There using hit rate as a DV in this case is meaningless, it doesn’t reflect any perceptual accuracy.

Question 5

What are the four types of responses that occur 2AFC

Answer 5

Stimulus present/respond present: Hit
Stimulus present/respond absent: Miss
Stimulus absent/respond present: False Alarm (FA)
Stimulus absent/respond absent: Correct Rejection

H+M= 100%
FA+CR= 100%
remember it’s within the stimulus presentation that responses must=100%

Question 6

What are the problems with using hit rate as a DV in 2AFC

Answer 6

o Omits the importance of correct rejections (CRs)
o Same with H-M (hits-misses)
o o If you start with sub-threshold sounds and progressively get louder, a person with tinnitus might respond “yes” to all/most of the trials and get a very high hit rate.
This is meaningless because it doesn’t actually tell u anything about their ability to discriminate the stimulus.

Question 7

Problems with reporting hit rate and false alarm as separate scores

Answer 7

 Widely used

 Difficult to interpret

Question 8

Problems with H-FA

Answer 8

 Still doesn’t fully dissociate the 2 processes but is better than % correct
 Won’t be more than 100%, so it is intuitive to interpret
• It penalises biases, for example the tendency to respond ‘present’ is penalised so interpretations of accuracy are not inflated the way they are when considering hit rate alone
• However, H-FA is not invariant under effects of varying bias
–> if the bias is made more liberal (more likely to say yes, stimulus is present) the hit rate can increase to 100%, but false alarm rate will also go up. So 100% hit - 10% FA gives a score of 90%. With high sensitivity it’s impossible to get a score of 100%, even though the high sensitivity is preferable in some cases such as bomb detection in airports.

Question 9

What is signal detection theory?

Answer 9

• • Signal detection theory described decisions made under uncertainty
• It separates out the effects of discriminability (d’) and bias (c’)
o You can change discriminability without changing bias and change bias without changing discriminability
• It distinguishes between different types of errors/success, and describe the trade-offs between them.
Model assumes that in perceiving a stimulus, the process has a component of noise
o This noise is assumed to be normally distributed
o There are different sources of this noise but these are not distinguished
o Thus, a fixed stimulus will seem to vary in perceptual strength (noise can increase the perceived strength of a present signal)
o In absence of this signal, perceived signal will be noise component
A perceived signal can therefore come from either signal present or signal absent
oJudgements are made based on perceptions

Question 10

Describe feature of SDT graph

Answer 10

• Model has 2 defining parameters:
o 2 normal distributions (shifted relative to each other) Further apart shows greater sensitivity or discriminability (easier task, better performance)
Distance between curves is d’ (standardised i.e. z function, same scale as cohens d)
o Point c’ (criterion, bias, k, beta) is a measure of bias
–>Criterion then provides a measure of bias
–> > c : signal present (to the right of criterion line)
< c : signal absent (to the left of criterion lint) •What we want to know:
o How far apart are the 2 distributions?
o Where is c’?

Question 11

How do you work out a z score?

Answer 11

z = (X - μ) / σ where z is the z-score, X is the value of the element, μ is the population mean, and σ is the standard deviation.

Question 12

What is the formula for d’ ?

Answer 12

d’ = z(H) -z(FA)

Question 13

What is d’ ?

Answer 13

• D-prime: distance between 2 distributions in unit of standard deviations
o Same scale as Cohen’s d
• Provides a measure of sensitivity (perceptual sensitivity or discriminability) on a scale that can be used across studies
• Widely used as a measure of perceptual sensitivity discrimination ability
• this is independent of bias

Question 14

What is c’

Answer 14

• C, the criterion that participants used to decide response
• Sometimes labelled beta (bias)
• If hit rate = false alarm rate, bias will be 0
Moves independently of d’

Question 15

How do you work out c’ ?

Answer 15

• C = -{average of z(h)z(FA)}
• Average ensures that the criterion point is in the middle of the signal and noise is given a bias value of 0
• Minus ensures that the value left of the criterion is negative (H>FA) and the value to the right of the criterion is positive (FA>H)

Question 16

Floor and ceiling effects in SDT c’ and d’

Answer 16

• Floor and ceiling effects produce problems
• For 0% and 100%, z-scores go to infinity and can’t be used
o Standard solution: use 1% and 99%
o Most scripts automatically do this, but be sure to check

Question 17

What is psychophysics?

Answer 17

• Aims to identify how varying a physical stimulus is associated with variations in perception and modelling this relationship

Question 18

Components of psychophysics experiment

Answer 18

• Stimulus
• Task (i.e. 2 AFC)
• Method
• Analysis
• Measure? Don’t have any examples or definition of this, but it may refer to performance based vs appearance based

Question 19

Name 2 methods in psychophysics experiments

Answer 19

1) Method of constant stimuli

2) Adaptive methods

Question 20

What is the method of constant stimuli

Answer 20

• Method of stimulus presentation in psychophysics experiment
Stimuli intensity is pre-defined
• Stimuli intensities are presented in (quasi) random order
• Typically used for appearance based experiments
o For performance based experiments, one should include stimuli intensities that straddle the expected threshold for 75% threshold, i.e. ranging from 50%-100%.
o For appearance based experiments this is also important but now the judgement is proportion (respones A)??????
• A fair number of trials is required at each stimulus intensity
• Will provide a more comprehensive measure of performance
• Typically used to compute measure of appearance as well as slope of psychometric function

Question 21

What is an adaptive method

Answer 21

Method of stimulus presentation in psychophysics experiment
• Stimuli intensity varies on the basis of performance
• Adaptive (e.g. staircase) algorithm is applied to choose optimal next stimulus
• Typically used to precisely identify the threshold (performance-based) –> Will provide a more precise estimate of individual threshold
• Faster because fewer trials are required
• Typically used to compute only the threshold (as opposed to slope of psychometric functions?)

Question 22

What are appearance based tasks?

Answer 22

• Appearance: indexes the stimulus intensity that is associated with equal probability of 2 responses
o Appearance based threshold is around 50% threshold
o Stimulus level chosen to cover full range of responses, avoid ceiling effects (0-100%)
• Type A vs B: are stimuli same or different?
• Single stimulus: left or right orientation?

Question 23

What are performance based tasks?

Answer 23

: index of how good someone is at a task
o Performance based threshold is around 75% threshold
o Stimulus level chosen to cover full range of responses, avoid ceiling effects (50-100%)

Question 24

What is a psychometric function?

Answer 24

• Function that relates behaviour on a task with some physical characteristics of the stimulus, e.g. contrast, length, duration
• We want to compute a threshold or an appearance-based measure
• A continuous function is fitted to the data
• We then identify the point on the function corresponding to a particular performance level
o 75% : threshold
o 50% : point of subjective equality

Question 25

What are the two parameters estimated when psychometric functions are fitted?

Answer 25

1) Alpha: determines position along the x-axis where the midpoint of the function corresponds to a value of interest (larger values reflect a rightward shifted function)
 Have to pre-specify what alpha is, toolbox will allow alpha to vary
2) Beta: steepness of the curve (higher values reflect steeper slopes)
 Beta can be referred to as slope, conceptually this is okay but technically it is not a slope
 People typically more interested in alpha values compared to beta

Question 26

List types of psychometric function

Answer 26

Logistic
Weibull
Gumbel
Cumulative normal
Hyperbolic secant
• Different functions can be applied, they typically have familiar sigmoidal shape
o Can specify what function to use, can change it but be make sure to tell readers about this, be explicit and ideally try to replicate it
• In most cases, choice of function may not be that crucial
• Most studies use logistic functions

Question 27

2 further parameters estimated when psychometric functions are fitted

Answer 27

1) Gamma: guess rate
o Takes into account expected accuracy if one were guessing
o This parameter is typically fixed in a model
2) Lambda: lapse rate
o Takes into account occasional attentional lapses
o This parameter is typically fixed in a model, but can be estimated if variable lapse rates across conditions may figure into the analyses

Question 28

How is goodness of fit of the psychometric function assessed in matlab?

Answer 28

In matlab, goodness of fit is reported by pDev [probability of deviance] and should be >.5 with larger values indicating superior fit. Provides a value of how much data deviate from fitted function

Question 29

Solutions to bad fit of psychometric function

Answer 29

• Possible solutions:
o Allow lapse rate parameter (lambda) to be free
o Remove particular stimulus levels and re-fit the function
 This is frowned upon/ not the conventional thing to do
Or remove particular participants who may have struggled with the task.
If you remove any data (participants or stimulus levels) you need to express this clearly in your results and state your reasons for doing so. You should show analysis with and without this data to show robustness of the effect. You might consider presenting excluded data separately in descriptive form for excluded data to show that it follows a similar trend even though the functions don’t fit.