Measurement, reliability + Validity Flashcards
Why is it important for psychologist to quantify everything?
- is a defining principle of science = measurement
- can be objectively obtained
- publicly available
- potentially checkable by others
What are the requirements for measurements?
- reliable = can be reproduced
- valid = actually measuring what we think it is measuring
What are the different levels of measurements?
- Nominal
- Ordinal
- Interval
- Ratio
Describe nominal level of measurement
- categorical. frq data
- when numbers are used as names (eg footballer numbers)
- All you can do is count how often each number occurs
Describe ordinal level of measurement
- numbers used as ranks
eg: order of finishing a race
Describe interval level of measurement
- scale with equal interval between points on a scale
- no true zero point
eg: IQ
Describe ration: level of measurement
- measurements made on a scale with equal interval between points on the scale
- scale has TRUE 0 point
Eg: reaction time, error scores
What is reliability?
- does it give consistent/ reproducible results?
Error in reliability may be due to….?
- natural phenomena variation
- -> within/ between individuals
- imprecision in defining + measuring psychological constructs
- -> what exactly is aggression?
What are different measures of reliability?
- Test-retest: time to time
- Alternate forms: version to version
- Split-half: item to item
- Inter-scorer: person to person
What factors affect reliability?
- phenomena itself
- -> trait vs state?
- Precision of measurements
- Sample size
- Variability in performance
- -> High > low
- Format of test:
- -> MCQ of 5Q = 25% correct by chance > than ….
- -> True/ False = 50% correct by chance
What is the replication crisis?
- many well-known findings aren’t that reliable
- -> small sample size
- -> Exact replication are rare
- -> “file drawer problem”
What is the “file drawer problem”?
- hard to get replication failures published
What are some solution to the replication crisis?
- replications
- Metal-analysis
- Converging operations = tackling problem with different methods, by different research groups
What are the 10 frq cited findings that have failed to replicate as found by Jarrett in 2016?
- power-posing
- Smiling = happier
- exposure to age related words = slower walk (x social priming)
- Babies are born with the power to imitate adults
What did Nosek et al in 2015 find when he replicated 100 experimental + correlational studies?
- og 97% significant results (p<0.5)
- replication = 36% significant results
- 47% replication significance size similar to og
= 68% studies w/ statistically significant effects
What does validity mean?
- are you measuring what you think you are measuring?
- a measure can be reliable without being valid (since you can replicate it i guess)
What link did Paul broca poorly attempt to make?
- anthropometric measurements + intelligence
- eg brain size saying there was a 14% difference between F +M but no regards for age/ body size given
What did Gall + Spurzheim which was scientific + testable but had poor vlaidilty?
- different parts of brain responsible for different functions
- highly developed regions = larger = reflected by bumps on skull
- craniometry measuring tool
What are the different types of validity?
- Face
- intuitively looks plausible - Content
- relevant material covered?? - Criterion
- predictive or concurrent - Construct
- does performance correlate well with known measures of the phenomena? - Ecological/ external
- relevance to rl
What does the predictive or concurrent criterion mean?
p: Does performance on test correlate w/ later performance?
C: does performance on my test correlate w/ other tests of current performance?
What factors affect the validity of experiments?
- good or bad experimental design
- method of difference: only one thing needs to differ between comparing conditions
What factors affect validity of tests?
- Norms + clear experimental standardisation