Quasi experimental designs

Question 1

What is quasi-experimental design?

Answer 1

• Approximate characteristics of true experiments
• What’s missing is random assignment
• Used when variable of interest cannot be manipulated
• Used when a naturally occurring variable is used to create comparison groups:
o PP variable: comparisons between PP types
 E.g. male v female, old vs. young
o Time variable: comparisons between different points of time
 E.g. before and after natural event
• Cannot infer cause and effect
• May have high external validity

Question 2

How to support causal claims for quasi experiment

Answer 2

• As with experiments, to support causality claims try to show:
o Cause precedes effect
o Cause co-varies with effect
o Alternative explanations are implausible

Question 3

Types of quasi experiment

Answer 3

Person-by-treatment

Natural experiments

Question 4

Person-by-treatment experiment

Answer 4

o Often conducted in lab
o Factorial design: at least 2 IVs
 One quasi-experimental (naturally occurring) variable
 Often pre-select PPs
 At least one manipulated independent variable with random assignment to manipulated variable
o Analyse quasi-IV like IV
o Cannot however infer cause-effect for quasi-IV in same way as for IV
o Effects of quasi-IV do highlight relationships
o Analysing quasi-independent variables can allow IVs to manifest their effects
o Where person variables are continuous or semi-continuous (as with IQ) rather than discrete, it may be better to treat them as a covariate than a quasi-IV

Question 5

Natural experiments

Answer 5

o ‘Naturally’ occurring manipulation
 e.g. compare similar groups of people before and after a random occurrence affecting groups differently
 e.g. lottery players: winners vs losers
 here, the compared groups should not differ in any systematic way
• Involves group comparisons
• Pre-test/post-test:
o Positive change in ‘treatment’ group (lottery winners) without change in ‘control’ group (lottery losers)
• Post-test only:
o ‘Treatment’ group exceed ‘control’ group

Question 6

Combination experiments

Answer 6

o Groups differ in individual characteristics and in natural manipulations
 E.g. measure person differences and wait for natural occurrences
 E.g. examine archive data on different groups of people before and after natural occurrences

Question 7

Threats to internal validity in natural experiments

Answer 7

General threats to internal validity
•	Individuals changing without treatment
o	History
o	Maturation
o	Testing
•	Non-treatment related differences between groups
o	Selection: non-random sampling bias
o	Maturation: similar groups grow apart
o	Mortality: differences between PPs dropping out from different groups

Other issues
• No control of quasi-independent variables
• Naturally occurring groups usually cannot be randomly assigned and are non-equivalent

Question 8

Time-series designs

Answer 8

Avoid some threats to internal validity in natural experiments
• Observe multiple times, introduce treatment, observe multiple times
• Can drop the control or comparison group (if you want?)
• Eliminate selection and selection by maturation threats- you can see the slope of how changes in the DV occur before the introduction of the treatment (which may be due to variables such as maturation), and you can see whether the introduction of the treatment leads to a change (instantaneous or delayed, continuous or discontinuous) in the intercept or trend of the slope.
• Assume effects of history, maturation, mortality, testing and other PP changes not due to treatment are consistent or cyclical over time

Question 9

Variations of time series analysis

Answer 9

o Reversible time-series
 Examine if there is a return to baseline after withdrawal of treatment
o Other variations sometimes possible
 E.g. introduce treatment at different times to different groups

Question 10

Threats to internal validity in time-series analysis

Answer 10

Threats to internal validity: there may have been other events that coincided with the naturally occurring treatment!!
o E.g. passing of new law such as speed restriction
• Still could be that ‘treatment’ coincided with other changes which caused the change
o E.g. petrol shortage resulting in people driving less
• Compare with control group in different state with petrol shortage but no speed restriction