Q2 reliability & validity Flashcards
what is true of a good measure?
it assesses behavioral variability accurately
observed score =
true score + measurement error
true score
systematic variance
measurement error
error variance
where does measurement error come from? (2)
participant
not the participant
2 ways participants may introduce measurement errror
transient factors
stable attributes
transient factors
anxiety, tiredness (not stable)
stable attributes
differ across participants but are stable within a given individual; IQ, personality traits, general motivation
3 ways (aside from the participant) that measurement error can be introduced
situation factors
measure factors
mistakes
situation factors
where are participants taking a questionnaire? lighting, temperature, etc.
measure factors
questions aren’t clear or the questionnaire isn’t good
what kind of mistakes can contribute to error variance?
participant is given the wrong test, experimenter says something wrong, computer error, etc.
reliability
consistency and dependability in scores across time; undermined by measurement error
how do we estimate reliability?
via correlations between measures of the same attribute
satisfactory reliability coefficient
0.7+ or 70%
3 kinds of reliability
test-retest
inter-item
inter-rater
test-rest reliability
will you get the same score of behavior if you measure at two different times? wouldn’t test non-stable traits
inter-item reliability
ensures consistency among scale items aiming to measure the same construct (internal consistency)
make sure there are high enough correlations between similar questions
number used to assess inter-item reliability and satisfactory threshold
chronbach’s alpha >0.7
what does chronbach’s alpha measure?
inter-item reliability
systematic variance
how do we assess inter-rater reliability in nominal data?
kappa coefficient (chi-square)
0.6+ is reliable
how do we assess inter-rater reliability in ordinal data?
spearman’s r (0.7+)
kendall’s tau (0.45+)
how do we assess inter-rater reliability in interval/ratio data?
pearson’s r (0.7+)
how do we assess inter-rater reliability in 3+ raters’ data?
intraclass correlation coefficient (0.75+)
