Diff in diff

Question 1

Example of cross-sectional v ITS?

Answer 1

Cross-sectional: compare # of banks in groups 6 and 8 in 1931

ITS: compare # of banks in group 9 in 1929 and then in 1931

Question 2

What assumption is critical for diff in diff?

Answer 2

Common trends

Question 3

How do we get the counterfactual trend?

Answer 3

Use trend (changes over time) in untreated group

Question 4

True or false: Selection bias related to fixed unobserved differences between T and U groups is ok

Answer 4

True, because we capture at both time points

Question 5

Outcome levels are not important; _____ are important

Answer 5

changes

Question 6

Write out model for two-group, two-time diff in diff

Answer 6

Yit=B0+B1Di+B2Post+B3(Di*post)+uit

Question 7

If the model is Yit=B0+B1Di+B2Post+B3(Di*post)+uit,

what does B0 represent?

Answer 7

The mean of the control group in pre-treatment period

Question 8

If the model is Yit=B0+B1Di+B2Post+B3(Di*post)+uit, what does B0+B2 represent?

Answer 8

The mean of the control group in post-treatment period

Question 9

If the model is Yit=B0+B1Di+B2Post+B3(Di*post)+uit, what is pre-treatment mean in treatment group?

Answer 9

B0+B1

Question 10

If the model is Yit=B0+B1Di+B2Post+B3(Di*post)+uit, what is the treatment effect?

Answer 10

B3

Question 11

If the model is Yit=B0+B1Di+B2Post+B3(Di*post)+uit, which term represents the selection bias?

Answer 11

B1

Question 12

How can you try to defend common trends assumption (3 ways)?

Answer 12

graph of pre-treatment trends
falsification test
controlling for time trends

Question 13

Model for generalized diff in diff, for statexyear panel where the treatment is turned on at different times for different groups

Answer 13

Yst=B0+B1(Treats*postt)+B2state+B3year+ust

Question 14

In this state by year model, what does B1 tell us? Yst=B0+B1(Treats*postt)+B2state+B3year+ust

Answer 14

How much, on average, outcomes differ in post period from that predicted by state and year fixed effects

Within-state changes over time in the outcome, for treatment and control states

Question 15

In this model, what does B3 tell us? Yst=B0+B1(Treats*postt)+B2state+B3year+ust

Answer 15

trends in the outcome common to all states

Question 16

Three approaches to choosing a comparison group?

Answer 16

• use all available non-T cases
• match on pre-T characteristics using propensity score
• geography

Question 17

True or false: It is a problem if you add controls to your diff in diff model and the estimates change

Answer 17

True–if adding controls changes estimates, you may have bad controls or non-random assignment to T (endogenous controls)

should only help reduce standard errors

Question 18

Why could you not just estimate a treatment effect by comparing the outcomes of treated units before and after treatment?

Answer 18

We might pick up the effect of other factors that changed around the time of treatment.

Fancy: Unable to distinguish between true effects and secular time trend changes.

Question 19

What is the common trends assumption?

Answer 19

Whatever happened to the control group over time is what would have happened to the treatment group in the absence of the program

Question 20

T or F You can test the common trends assumption

Answer 20

False, we can never observe the true counterfactual (how the outcome would have changed over time without treatment)

Question 21

Scenario: In what direction would the following be biased: Parallel trends assumption is violated because the outcome was already rising faster in the treatment group than in the control group

Answer 21

The D-in-D estimate would over estimate the impact of the treatment.

Question 22

What is one way to relax the parallel trends assumption?

Answer 22

To allow for group or unit-specific time trends

Question 23

How does the inclusion of unit-specific time trends change the interpretation of a diff-in-diff model?

Answer 23

Removes unit specific linear time trends from the outcome, treatment, and covariates.

Allows treatment and control states to follow different trends.

B1 now identified off of “deviations” from the group-specific common trend in the outcome of interest.

Question 24

Model for group-specific diff in diff, for state-specific time trends

Answer 24

Yst = B0 + B1 (TREAT_s * POST_t) + B2STATEks + B3YEARjt + B4(STATEks * t) + ust

t = linear time trend (as opposed to YEAR which allows effect to vary by year)