Chapter 11 Regression discontinuity designs

Question 1

1. How is treatment assignment performed in a regression discontinuity design?

Answer 1

based on a cutoff score on a continuous pre-treatment measure (i.e., a single observed confounder)

Question 2

2. What is the difference between the treatment assignment mechanism for a regression discontinuity design and the treatment assignment mechanism for a propensity score matching design?

Answer 2

RD: The regression-discontinuity (RD) design is a quasi-experimental design where participants are assigned to treatment or control groups based on a cutoff score on the pre-treatment measure (i.e., a single observed confounder).

Matching: It is advantageous to match on the linear propensity score (i.e., the logit of the propensity score) rather than the propensity score itself, because it avoids compression around zero and one.

(Unlike experimental designs, RD design does not require us to assign individuals in need of treatment to a no-treatment control group in order to evaluate the effectiveness of a treatment.)

Question 3

3. What is the rationale for the use of the regression discontinuity design to obtain unbiased treatment effect estimates?

Answer 3

RD does not suffer from selection bias because it is reasonable to assume that in the absence of the treatment, there would be no discontinuity between the two groups.

Question 4

4. What types of variables can serve as an assignment variable in a regression discontinuity design?

Answer 4

The assignment measure can be the same or different as the outcome measure.
It can be a complex variable:
A composite variable.
A Combination of cutoffs on multiple variables

Question 5

5. What is one advantage of the regression discontinuity design over an experimental design?

Answer 5

Unlike experimental designs, RD design does not require us to assign individuals in need of treatment to a no-treatment control group in order to evaluate the effectiveness of a treatment.

Question 6

6. What is one limitation of the regression discontinuity design over an experimental design?

Answer 6

RD designs are not as statistically powerful as randomized experiments. [Randomized experiments more efficient than RDD by a factor of about 2.75 (Goldberger, 1972b). Therefore, in order to achieve the same level of statistical accuracy an RD design needs as much as 2.75 times the participants as a randomized experiment.]

Question 7

7. What would be a threat to the internal validity of a regression discontinuity design?

Answer 7

Threat to internal validity:
The only threat to internal validity in RD designs would be something that would cause a discontinuity in the regression lines at the cutoff.

A threat to internal validity would have to be treatment related (for example, treatment-related mortality).

Question 8

8. What would be a threat to the statistical conclusion validity of a regression discontinuity design?

Answer 8

Threats: Effects are unbiased only if the functional form of the relationship between the assignment variable and the outcome variable is correctly modeled, including:
Nonlinear Relationships
Interactions

Question 9

9. Why does the regression discontinuity design have lower external validity than an experimental design?

Answer 9

An RD design estimates the average treatment effect at the cutoff while a randomized experimental design estimates the average treatment effect.

The effect at the cutoff only generalizes to the specific cutoff with the assignment variable used.

Question 10

10. What is the difference between sharp and fuzzy regression discontinuity designs?

Answer 10

Sharp RD designs: Requires that the probability of treatment changes discontinuously from 1 to 0 at the cutoff .
Treatment effects are identified at the cutoff

Fuzzy RD designs: Probability of treatment is different of 1 at the cut-off.

Treatment effect estimation is still possible for those close to the cutoff that are given the treatment .

Question 11

11. What is the difference between type I and type II fuzzy regression discontinuity designs?

Answer 11

Type I fuzzy design: some in the treatment group fail to receive the treatment (referred as no-show)

Type II fuzzy design: Some in the treatment group do not receive treatment and some in the control group receive treatment (referred as cross-over)

Question 12

12. What is the simplest ANCOVA model for data from a regression discontinuity design?

Answer 12

When the relationship between pretest and posttest scores is linear, an ANCOVA model can be used for analysis: (look at formula and explanation in ppt)

Question 13

13. What are the assumptions of an ANCOVA model for data from a regression discontinuity design?

Answer 13

When the relationship between pretest and posttest scores is linear

Question 14

14. What is the role of centering in ANCOVA analysis of regression discontinuity design data?

Answer 14

Centering the assignment measure around the cutoff is accomplished by subtracting the cutoff from all assignment measure scores.

This is useful because it causes the RD equation to estimate the effect of treatment at the cutoff score, which is the point where groups are most similar.

After centering assignment scores around cutoff, the intercept of the regression is the outcome at the cutoff value for the untreated group.

Question 15

15. What is the main advantage of local linear regression for analyzing regression discontinuity design data over ANCOVA?

Answer 15

It allows the researcher to relax assumptions about the functional form of the relationship.

Question 16

16. What is the role of a bandwidth in local linear regression?

Answer 16

A long bandwidth can produce biased estimates but a short bandwidth will reduce efficiency.

Question 17

17. What is the main advantage of a regression discontinuity design with multiple cutoffs rather than with a single cutoff?

Answer 17

The purpose of this design is to increase the generalizability of the local average treatment effect (LATE) by comparing the effects across multiple scores of the assignment measure.

Question 18

18. How can an ANCOVA model for single cutoff regression discontinuity design be used to analyze a design with multiple cutoffs?

Answer 18

?Convert design to a 2 dimensional RD:

For each cutoff, center assignment variable atzero.

Poolobservationsintoasingle dataset.

Analyze asatwo dimensional RDwithasingle assignmentvariableand acutoff atzero.

Estimateusingpooled data and a parametric RD model.

Question 19

19. What are the assumptions necessary to use an ANCOVA with a multiple cutoff regression discontinuity design?

Answer 19

?Each cutoff group has the same functional form;

There are constant treatment effects across cutoffs (though treatment by cutoff interaction terms could be included);

Problem: Pooling across multiple cutoffs sometimes obscures sources of heterogeneity because cutoffs often vary by sites, time, cohorts, grades;

Question 20

20. What is the simplest multilevel model for data from a regression discontinuity design with multiple cutoffs?

Answer 20

This is a multilevel model with site-specific cut-off centering (see formula in ppt)
Models for meta-analysis can also be used.

Question 21

21. What is one advantage of using multilevel modeling rather than ANCOVA to analyze a regression discontinuity design with multiple cutoffs?

Answer 21

?
Solves problem of pooling acrossmultiple cutoffs sometimes obscures sources ofheterogeneitybecause cutoffs often varybysites,time, cohorts, grades

Question 22

22. How can the regression discontinuity design be combined with a randomized experimental design?

Answer 22

[forumal in ppt]
Defining Xc as the median of the interval will increase the power of the test.

Defining Xc as the cutoff that assigns all participants in the treatment group to the same side of the cut-off helps estimating an interaction between treatment and assignment score more accurately.

Question 23

23. What are the advantages of combining the regression discontinuity design with a randomized experimental design?

Answer 23

It increases the power of RD, with an increase proportional to the range of the randomization interval relative to the total range of the assignment variable.

It allows to estimate regressions for treated and untreated within the randomization range, which helps with defining the functional form of the relationship between Y and X.

It is a practical method when a clear choice for a single cut-off does not exist.