Lecture 11 Flashcards
Whats one pressing issue with the potential outcomes framework?
Two potential outcomes exist, but only one is observed for the same individual, making causal inference hard
- so we need to infer causal effect by comparing treated and untreated individuals econometrically
ATE - average treatment effect: E [ Y1 - Y0 ]
- represents the causal effect of treatment on entire population
ATT: Average treatment effect: E [Y1 - Y0|X = 1]
Represents average causal effect on those who received treatment
What does an OLS regression of Y on binary X estimate?
The difference in means:
- B = E[Y|X=1] - E[Y|X=0]
- Cov(X,Y) = p(1-p) x (E[Y|X=1] - E[Y|X=0])
Causation vs Selection:
- recall when X = 1, Y = Y1, and when X = 0, Y = Y0
B = E[Y|X=1] - E[Y|X=0], simply the difference in mean outcomes between treated and control groups
= (E[Y1|X=1] - E[Y0|X=1]) + (E[Y0|X=1] - E[Y0|X=0])
- first bracket is the ATT, causal effect of treatment on those who actually received it
- second bracket is selection bias, measuring how untreated outcomes differ between the two groups
How does the assumption of mean independence affect this regression?
The selection bias = 0, therefore B = ATT
- if y1-y0 is also mean independent of X then E[y1-y0|x=1] = E[y1-y0], so ATT = ATE
How do you even interpret E[Y0|X=1]
- what about E[Y1|X=0}?
Counterfactual, essentially saying the potential outcome without treatment for those who were treated
- also counterfactual, what is the potential outcome for those without treatment, if they actually received treatment.
How can random assignment affect selection bias
- random assignment means no selection bias, as individuals do not self-select into groups based on their expected outcomes
- no selection bias is sufficient for mean-independence.
What do we need to do when randomisation is based on covariates?
Need control variables
- case 1 - if randomisation is purely random, and independent of covariances, control variables are not needed for unbiasedness
- case 2 - if randomisation is not purely random, and control variables are introduced improperly, can introduce selection bias as they can correlate with treatment assignment
Selection bias may arise when y0 is not mean independent of x, as both y0 and x related to a 3rd variable, a covariates.
- e.g.
Study of effects of extra tutoring X on student marks Y, suppose students in higher performing cohorts are more likely to get tutoring
- W - cohort performance affects both
How to solve this selection bias due to a covariate?
Unconfoundedness condition - after accounting for W, treatment and control groups are comparable in terms of potential outcomes
Sometimes, group level factors influence both the treatment and the outcome, so if we ignore them, our results can become biased, as we are mixing up differences between groups with the actual treatment effect
- how to fix?
Control for group level dummies by including group dummies, variables which indicate which group each individual belongs to
Weights of OLS with group fixed effects
B1 is not the population average causal effect, rather is a weighted average of group specific causal effects, but weights depend on both group size and treatment probability
Intuitively, larger groups and groups with a balanced mix of treated and untreated contribute more to B1.
Pros of experiments
- randomisation eliminates the selection bias
- same average effects for treated and untreated groups
- OLS identifies causal effects, internally valid as causal conclusions apply to the population in they experiment
Threats to internal validity
- Failure to randomise
- Partial compliance with the treatment protocol
- Spillover effects
- Attrition
Failure to randomise
Treatment and control groups may not be wholly comparable as differences in outcomes may not be attributed solely to the treatment
Tests for randomisation
- Regress X on a set of predetermined variables W, test via an F test whether all coefficients on W are jointly equal to 0, if X is unrelated to W, treatment assignment likely random
- Or do a placebo test, where you regress predetermined variables W on X, they shouldn’t systematically differ between treatment and control groups if randomisation is successful
Partial compliance with the treatment protocol
Let’s say observers were assigned to certain areas, but decided not to comply, leads to attenuation of the differences between the two groups
Solution - can you track compliance, and then use instrumental variables to fix
Spillover effects
When defining potential outcomes, typically assumed that the outcome for unit i onto depends on xi, not on the treatment status of other units, but if violated - spillover effect
- i.e observers in one polling station may lead to reallocation of fraud to a nearby one.
Attrition
Attrition is when some units drop out of the sample during the experiment or before the final outcome is measured
- if decision to leave is related to potential outcomes - creates selection bias, and violates assumption of random assignment.
Why are experiments not everywhere?
- costly
- may raise ethical/ political economy concerns
- not fully informative on the effects of the policy of interest
