Methods

Question 1

Causality

Answer 1

X → Y, i.e. the correlation is due to X causing Y .

Question 2

Reverse causality

Answer 2

Y → X, i.e. Y causes X.

Question 3

Simultaneity

Answer 3

X → Y and Y → X, X and Y are jointly determined.

Question 4

Spurious correlation/omitted variable bias

Answer 4

the correlation is due to the fact that there exists a third variable, Z, such that Z → X and Z → Y , i.e. both X and Y are affected by Z.

Question 5

ATE (Average Treatment Effect) full compliance

Answer 5

ATE = Average Outcome in Treatment Group - Average Outcome in Control Group (in RCT)
Can examine heterogeneous treatment effects by comparing subgroups of treated and control individuals
ATE = (Yt1-Yt0)-(Yc1-Yc0) (in diff-in-diff)

Question 6

External validity

Answer 6

To what extent do the results apply to populations beyond the sample used in the analysis

Question 7

ITT (Intention-to-treat effect) partial compliance

Answer 7

If we randomize who gets treated and who not, but not everyone in the treatment group participates/takes up the treatment, i.e. the take-up rate, or compliance rate, is <100%,

Question 8

Hawthorne effect

Answer 8

individuals in the treatment group may change their behavior because they are aware they are being studied
E.g. participants may pay more attention, experimenter demand effect

Question 9

John Henry effect

Answer 9

individuals in the control group may change
their behavior, e.g. they may work harder to compensate for not being treated
If this is the case, the treatment effect will be underestimated

Question 10

Contamination or spillover effects

Answer 10

This happens when some of the controls that were
randomly excluded from treatment are able to receive the treatment, or, conversely, if some members randomly selected for the treatment do not receive it.
To avoid this, treatment and control need to be kept sufficiently separate, making similarity between the two groups also more difficult to achieve.

Question 11

Dropout or attrition effects

Answer 11

Some members of the treatment group may drop out
of the experiment before completing the program. This is often the case for experiments that last for a long time. In this case, there is a selection effect since randomization was done on the offer of treatment or at the outset of the experiment, not on whether individuals completed the treatment. Selection in dropping out may create bias if those who drop out are different in terms of the measured outcome from those who continue the experiment.

Question 12

Exclusion restriction

Answer 12

The instrument must not be correlated to the outcome through another channel

Question 13

Counterfactual

Answer 13

The counterfactual should measure what would have happened to the beneficiaries in the absence of the intervention.

Question 14

Write down the empirical model

Answer 14

the sign of the bias depends on how Z (the omitted variable) and X (the potentially endogenous variable) and how Z (the omitted variable) and Y (our outcome of interest) are correlated:
If both are positively (or negatively) correlated, our coefficient estimate on X will have an upward bias / will be biased upward
If one is positively and one is negatively correlated, our coefficient estimate on X will have a downward bias / will be biased downward
In the case of impact evaluations, X is usually expected to have a positive effect on Y
If that is the case and if the estimate is biased upward, we overestimate the positive effect of X on Y

Question 15

Stratification

Answer 15

Randomization ensures that treatment and control groups will be similar in expectation
Stratication is used to ensure that the two groups are similar in practice

Question 16

Selective attrition

Answer 16

Attrition refers to the case when we lose observations (individuals,firms, etc.) over time, i.e. between baseline and endline

Question 17

Internal validity

Answer 17

anything that could harm an experiment itself, i.e. the effect on treatment dummy may not reflect the causal effect of the program

Question 18

Oversubscription method (Alternative Methods of Randomization)

Answer 18

when there is too much demand, use of the lottery system, i.e. there are more applicants than spots, or if it faces budget or capacity constraints

Question 19

Randomized order of phase-in (Alternative Methods of Randomization)

Answer 19

Potential disadvantage is that one cannot examine long-term effects
Problematic if control group is affected by the expectation of future treatment

Question 20

Parallel trend assumption

Answer 20

in the absence of the program, the change in the outcome variable among beneciaries would have been identical to that among non-beneciaries (diff-in-diff)