validity & reliability Flashcards
Test-retest reliability
- Do a test today and then a month later - would you get the same results?
- Within close time frame
Internal consistency
2 different versions, counterbalance
- How correlated are they? Are they measuring the same thing -> high correlation
Interrater reliability
- Subjective
- Researchers code elements from ppts responses
- Qualitative research -> derive numbers
- Different observers should rate consistently
- Get rid of individual biases
Consistent score -> reliable
Are our measures reliable?
- Test-retest reliability (consistent across times)
- Test-retest correlation, Bland-Altman plot
- Internal consistency (consistent across items)
- Split-half correlation, Cronbach’s α (alpha)
- Interrater reliability (consistent across researchers)
- Intraclass correlation, Cohen’s k (kappa)
Construct validity
- How well does a measure relate to the theoretical concept you are studying?
- How well do set of questions relate to what you want to measure e.g. anxiety questionnaire actually assess anxiety?
Three broad components of Construct Validity
- Describing theoretical concepts and relationships
- Developing measures of constructs proposed based on theory
- Empirically test the relationships
(Cronbach & Meehl, 1955)
Content validity
- How well does a measure relate all facets of a given construct
- E.g. both positive and negative symptoms of schizophrenia
Internal validity
- How well does a measure account for the manipulated change?
- E.g. strength of causal relationship, potential influence of confounding factors
- Does it affect the relationship or does something confound it?
Examples of factors that can lead to a decrease in internal validity (threats)
- Participant selection - should be randomised and counterbalanced
- Participant motivation (fatigue, time of day)
- Maturation (development, aging)
Examples of factors that can lead to a decrease in internal validity (threats) ii
- Experimenter training
- Equipment decay/use
- Lack of random assignment
External validity
- Generalisation of findings
- How well do findings relate to conceptually similar circumstances?
- High external validity (such as a field experiment) may lead to less control for confounds
Controlled experiment validity
low in external validity, high in internal validity
Field exp. = naturalistic
validity
high in external validity, low in internal validity -> hard to control for confounds
Population validity
- Many studies are based on undergraduate students, how well do these findings generalise to other populations?
Has been suggested that most of the psychology literature is based on a WEIRD demographic
