Week 7 Flashcards
What is the evaluation problem?
Determining causality, e.g: did the policy cause the desired outcome or was this due to other influences.
What is a counterfactual?
It is the outcome in the world where the intervention did not happen, allowing for comparison.
What are the 3 assumptions for a valid comparison group?
-Average characteristics between treated and control groups.
-Treatments and controls must’ve reacted the same in the presence of treatment.
-Treatments and control groups not exposed to different interventions.
What are the before-after comparisons?
The counterfactual is the outcome for the same individual prior to the intervention… assuming that the outcome for the individual would have been the same as it was before the intervention (main limitation as this is rarely the case).
What is the treated vs non-treated method?
Compare those who were and were not treated, where the counterfactual is the case for individuals who do not enrol in the program. However, problems will arise as those who enrol may have different characteristics in comparison to those who do not enrol.
What is the randomisation solution?
To distribute resources evenly among equally deserving populations… creating the counterfactual of a comparison group of similar characteristics on average. The only remaining difference between the groups will be participating in the training programme. However, potential disadvantages are that it is hard to generalise the result to other populations due to differences and there may be other issues, such as retrospective evaluation (the programme has already finished), non-excludable national policies, being unfeasible to randomise and ethical reasons.
How can the DID model be helpful in impact evaluation?
It can explore changes in outcomes over time between non/treated groups.
It combines the two estimated outcomes to provide an overall comparison which is more accurate.
The counterfactual is the trend in outcomes for the comparison group.
What are the 3 requirements for the use of the DID model?
-Constant characteristics of the two groups.
-A baseline data.
-The difference in outcomes over time of the treatment and comparison groups would have changed at the same rate in the absence of the program.
What is Ashenfelter’s dip?
Participants of the training program often experienced a dip in earnings prior to joining the programme, causing an upward-bias of the DID estimate of the programme.
What is the role and requirements of an instrumental variable (IV)?
To account for endogeneity in individual participation and reverse causality, meaning the causal estimates are more convincing.
They are required to be highly correlated to the program participation but not to the unobserved characteristics that affect the outcome of interest.
What is the regression discontinuity design?
A non-experimental impact evaluation method that creates a counterfactual from an exogenous eligibility rule of programme participation.
What are the 2 conditions for the RDD?
-There’s a continuous eligibility index in which individuals can be ranked.
-There is a clear, exogenous threshold where you can be considered eligible.
What is matching methods?
It is a non-experimental impact evaluation method that uses statistical techniques to construct an artificial counterfactual group.
What are the advantages and disadvantages of the matching method?
-Can be applied in almost any context as long as there are non-participants.
-However, there may not be enough data on the matching group, no comparable non-participants and observed factors alone must determine the program participation (a strong assumption).
