Lecture 12 - Threats to Validity Flashcards
Define ‘internal validity’
The extent to which the results obtained are a function of the variables that were systematically manipulated
- Are changes in the DV caused by changes in the IV?
- Does this study measure what its intending to measure (the IV)
High internal validity - strong evidence of causality
Low - little or no evidence of causality
How can we maximise internal validity?
Must be able to rule out the possibility of confounding variables (factors other than the IV producing a change in the DV)
Therefore there must be a high amount of control and elimination of potential exteraneous influences
Easiest to do in a highly controlled, laboratory setting
Name 6 threats to internal validity
history effects maturation effects mortality instrument decay participant selection statistical regression to the mean
What are history effects and how do we solve them?
Events occurring during the experiment that are not part of the treatment - eg a fire alarm goes off at 11am and affects all groups in at that time but not all groups in later or earlier conditions
Solution: hold experiences as constant as possible except for the IV. this includes randomly assigning conditions to time - you don’t want all of condition 1 to be in the 11am group!
Wha tare maturation effects
processes within participants that may simply change due to the passing of time - such as fatigue, hunger, and in the long term, ageing.
What is mortality and how do we solve it
not neceesarily death! Could just be people who drop out.
The people who drop out may well be different from those who do not, invalidating the study
So this is why we tend to provide an inconvenience allowance, so people are unlikely to drop out
What is instrument decay and how do we solve it?
can be equipment - becomes less accurate with age
or experimenters - become more skilled with time or become more bored with time
Solution: randomise allocation of experimenters to time slots, train the staff, calibrate instruments regularly
What is participant selection and how do we solve it?
eg only males in control group but both genders in experimental group - obviously going to affect the result
Solution: random allocation or matched pairs
What is statistical regression to the mean?
going back to the mean after extreme behaviour - so if someone was selected for treatment because of their extremely low score on a depression test, they may well regress to the mean naturally and you cant tell if this is due to the treatment or just regression to the mean
What effects occur that equalise groups when participants are allowed to communicate?
Diffusion of treatment effects
Compensatory rivalry - participants who become aware theyre in the control group work harder to overcome this ‘disadvantage’ - also called the John Henry effect
Compensatory equalisation - experimenters aware of conditions make up for what they feel is a disadvantage, eg control group, less effective treatment, and perform things above what they are supposed to do.
What effects occur that separate groups when participants are allowed to communicate
resentful demoralisation - if control group is aware they are the control group then they may give up and not try very hard, increasing the difference between the groups (but not as a result of the IV)
What is external validity?
For an IV - Does it accurately measure the phenomenon?
For an experiment - does it represent the real world? e.g. field experiments (but often comes with a lack of control)
what is population validity
the extent to which the results can be generalised from the experimental sample to the population as a whole
what is ecological validity
the extent to which the results can be generalised from the set of environmental conditions in the experiment to other environmental conditions
Name 4 threats to external validity
reactive effects of testing
reactive effects of experimental setting
selection-treatment interaction
multiple treatment interference
