CHAPTER 12 Regression Discontinuity Designs Flashcards
What is a regression discontinuity design?
A method to estimate causal effects when a treatment changes discontinuously at a known threshold.
What does regression discontinuity designs estimate?
A local average treatment effect for units right around the threshold where treatment changes.
In what situation is a regression discontinuity design appropriate?
When a treatment changes discontinuously at a known threshold.
What is the running variable in a regression discontinuity design?
A third variable that determines treatment assignment based on whether it is above or below a threshold.
What is the local average treatment effect (LATE)?
The average treatment effect for units with values of the running variable right at the threshold.
True or False: Regression discontinuity designs can provide unbiased causal estimates.
True.
What is an example of a situation where a regression discontinuity design might be used?
Estimating the effect of receiving a merit scholarship based on a strict scoring rule.
What is the naive approach in implementing an RD design?
Comparing mean outcomes for small ranges of the running variable on either side of the threshold.
What is a significant disadvantage of the naive approach?
It is likely to produce biased estimates due to correlation between the running variable and potential outcomes.
What is a local linear approach in regression discontinuity designs?
Running linear regressions of the outcome on the running variable separately on either side of the threshold.
What is the purpose of selecting a bandwidth in an RD design?
To balance the trade-off between reducing bias and improving precision.
What does the regression discontinuity design estimate when comparing groups just above and below the threshold?
The ‘jump’ or ‘discontinuity’ in the outcome as the running variable crosses the threshold.
What is the effect of using polynomial regressions in an RD design?
It accounts for possible non-linear relationships between the running variable and the outcome.
Fill in the blank: The regression discontinuity design is useful for estimating effects of government programs that change at known _______.
thresholds.
What is an important consideration when interpreting results from a regression discontinuity design?
The estimated effect is local and may not represent the overall average treatment effect in the population.
What is one common application of regression discontinuity designs?
Estimating the effects of government programs that are means-tested.
True or False: The local average treatment effect can differ from the overall average treatment effect.
True.
What is the significance of the threshold in a regression discontinuity design?
It determines which units receive the treatment and which do not.
What does it mean when we say the average treatment effect can differ at different values of the running variable?
The effect of treatment may vary for different groups near the threshold.
What is a common way that people implement RD designs?
Polynomial regressions that regress the outcome on treatment, the running variable, and higher-order polynomials.
What is a downside of using polynomial regressions in RD designs?
Data points far from the threshold can have a big effect on the estimated discontinuity.
What should researchers avoid when implementing an RD design?
Over-comparing and under-reporting.
What factors should decisions in RD designs depend on?
Substantive knowledge, beliefs about the relationship between running variable and outcome, and bias-precision trade-off.
What does it indicate if estimates are robust across different bandwidths in RD designs?
It lends additional credibility to the results.
What is the running variable in the analysis of political connections?
The vote share of the politically connected candidate.
In the RD analysis of political contributions, what is the treatment?
An indicator for whether the candidate won the election.
What is the effect of smaller bandwidths on estimates in RD designs?
Estimates become more volatile and confidence intervals are larger.
What does a large negative discontinuity at the threshold indicate?
Nominating an ideological extremist significantly decreases a party’s performance in the general election.
What does continuity at the threshold ensure in RD designs?
Baseline characteristics do not change discontinuously at the threshold.
What happens if continuity does not hold in an RD design?
The RD will give a biased estimate of the local average treatment effect (LATE).
What is the primary concern when comparing electoral fortunes of extremists and moderates?
Establishing an unbiased estimate of the effect of nominating an ideologically extreme candidate.
What is the fundamental problem of causal inference mentioned in the text?
It’s impossible to say with confidence what would have happened under different circumstances.
In the context of the Democratic party, what was the debate surrounding the electability of candidates?
Whether extremist candidates or moderate candidates are more electable.
What does the term ‘continuity at the threshold’ refer to?
The requirement that baseline characteristics do not jump at the threshold.
What is the implication if the estimates change meaningfully as the bandwidth increases?
There is a trade-off between bias and precision.
What did Andrew Hall’s 2015 study find about ideological extremists in elections?
Nominating extremists, on average, is a bad electoral strategy.
What does the analysis of congressional elections suggest about ideologically extreme candidates?
They may not correlate positively with electoral success.
What is the effect of nominating an extremist candidate on voter turnout according to Hall and Thompson’s follow-up study?
Extremist candidates may turn out the wrong base, motivating opposition voters.
What is necessary for RD designs to provide unbiased estimates?
The treatment status must change sharply at the threshold without other influencing factors.
Fill in the blank: The average treatment effect at the threshold is the difference between the _______ and the hollow dots.
filled-in gray dots
What does it mean if the average potential outcomes show no jump at the threshold?
Continuity is satisfied, allowing for unbiased estimates.
What is the significance of comparing electoral outcomes for different candidates?
To assess the impact of candidate ideology on electoral success.
According to the text, what can affect the running variable in RD designs?
Units have control over their value of the running variable.
What is the continuity assumption in regression discontinuity designs?
It assumes that potential outcomes are continuous at the threshold without any discontinuities in baseline characteristics
Continuity does not require that the outcome be unrelated to the running variable or that units have no control over their value of the running variable.
What could violate the continuity assumption in regression discontinuity designs?
- Precise control over the running variable leading to clustering around the threshold
- Manipulation of scores or thresholds by the committee
- Other characteristics changing at the threshold
For example, privileged students might have better information about scoring systems.
What is a real-world example of continuity violation mentioned in the text?
Mayoral salaries in France that change at a population threshold of 3,500 residents
Other governance features also change at this threshold, affecting the results.
What should be assessed before interpreting results from a regression discontinuity design?
The plausibility of the continuity assumption
This can be done by knowing details of the situation or analyzing pre-treatment characteristics.
How can one validate the continuity assumption?
- Analyze pre-treatment characteristics for discrete jumps at the threshold
- Assess the distribution of the running variable for bunching
Bunching indicates potential manipulation that violates continuity.
What is the implication of a small violation of the continuity assumption?
The regression discontinuity estimate may be biased, but possibly only slightly
The degree of bias depends on the precision of sorting around the threshold.
What is a sharp regression discontinuity design?
A design where treatment is completely determined by the running variable and the threshold
It contrasts with fuzzy regression discontinuity designs.
What does fuzzy regression discontinuity design involve?
Combining regression discontinuity with an instrumental variables approach due to noncompliance
Treatment may be affected by the threshold but not deterministically.
In the Vietnam War example, what was the goal of the bombing campaign?
To suppress Viet Cong guerilla forces in northern Vietnam
The effectiveness of this strategy was evaluated using regression discontinuity design.
What was the Hamlet Evaluation System (HES)?
A system that collected data to evaluate hamlet security during the Vietnam War
Scores were rounded to letter grades that determined bombing targets.
How did the discontinuity in the Hamlet Evaluation System affect bombing decisions?
Hamlets just below the threshold received more bombings than those just above
The perception of security differed based on rounded letter grades.
What is the significance of the example regarding hamlets bombed in Vietnam?
It illustrates how regression discontinuity can be used to analyze causal effects in complex situations
The analysis revealed that hamlets with slightly better grades were bombed less frequently.
What is the main concern regarding continuity in electoral regression discontinuity designs?
Manipulation of election results in close elections
If parties can influence outcomes, continuity may be compromised.
What evidence suggests continuity issues in U.S. House elections?
Incumbent parties are statistically more likely to win elections decided by less than 0.25 percent of the vote
This raises concerns about manipulation affecting electoral outcomes.
What do Devin Caughey and Jas Sekhon argue about electoral manipulation?
Evidence suggests incumbents can strategically influence close elections
However, they also question the precision of electoral forecasts.
What is the relationship between the running variable and treatment in fuzzy regression discontinuity?
The side of the threshold determines treatment assignment, but not all units comply
This leads to the need for instrumental variable methods.
What does a regression discontinuity (RD) design estimate?
A causal effect that estimates the discontinuous jump in an outcome on either side of a threshold that determines treatment assignment.
What is the running variable in an RD design?
A variable for which units’ treatment status is determined by whether their value of that variable is on one or the other side of some threshold.
What is meant by continuity at the threshold in RD designs?
The requirement that average potential outcomes do not change discontinuously at the threshold that determines treatment assignment.
What is a sharp RD?
An RD design in which treatment assignment is fully determined by which side of the threshold the running variable is on.
What is a fuzzy RD?
A research design that combines RD and IV, used when treatment assignment is only partially determined by which side of the threshold the running variable is on.
What are the four conditions required for an instrument in an IV approach?
- Exogeneity
- Exclusion restriction
- Compliers
- No defiers
What is the local average treatment effect (LATE)?
The effect of treatment on the outcome for units whose treatment status is affected by the instrument at the threshold.
What does the analysis by Dell and Querubin find about bombing and insurgency?
Bombing was counterproductive; more bombing increased the probability of Viet Cong activity.
True or False: The treatment in the RD design is binary.
False
What was the outcome of interest in the analysis of hamlets and bombing?
Viet Cong activity in the hamlet following the tabulation of the scores.
How does the concept of noncompliance affect RD designs?
It indicates that treatment status may not solely depend on the threshold, leading to issues in estimating causal effects.
What is the significance of the threshold in RD designs?
It allows researchers to compare outcomes just above and below the threshold to estimate causal effects.
What did Jonah Berger and Devin Pope analyze using an RD design?
The effect of psychological motivation on performance in professional basketball games.
In the basketball study, what was the running variable?
The point margin of the home team at halftime.
What was a surprising finding from Berger and Pope’s RD analysis?
The home team was more likely to win when they were 1 point behind at halftime than when they were 1 point ahead.
Fill in the blank: A fuzzy RD design is used when treatment assignment is only _______ determined by the threshold.
[partially]
What is the main concern when using the exclusion restriction in RD designs?
Whether the running variable has any effect on the outcome other than through its effect on treatment.
What happens if continuity at the threshold does not hold?
A regression discontinuity design does not provide an unbiased estimate of the local average treatment effect.
What is the local linear approach in RD design?
A method that regresses the outcome on the running variable to estimate the effect at the threshold.
What is one potential limitation when estimating effects near the threshold?
The choice of bandwidth can significantly affect the estimates.
What is the treatment in the context of Pell Grants?
College attendance.
What is the running variable for estimating the effect of Pell Grants?
Family income.
What would you need to ensure to obtain credible estimates in an RD design?
Assumptions about the continuity at the threshold and the validity of the instrument.
What is the focus of the study by Andrew C. Eggers et al. (2018)?
Potential violations of continuity in studies of policy changes at population thresholds
Discusses pitfalls and solutions for regression discontinuity designs based on population thresholds
What does the study by Andrew B. Hall (2015) investigate?
The effects of electing an extremist versus a moderate in a primary election
Analyzes the outcomes when extremists win primaries
What do Caughey and Sekhon (2011) study regarding electoral designs?
The validity of electoral regression discontinuity designs
Focuses on lessons from close U.S. House races from 1942 to 2008
What is the contribution of Eggers et al. (2015) to electoral designs?
New evidence on the validity of the regression discontinuity design for estimating electoral effects
Analyzes over 40,000 close races
What is the main subject of the study by Melissa Dell and Pablo Querubin (2018)?
U.S. bombing during the Vietnam War
Examines nation building through foreign intervention and military strategies
What phenomenon do Berger and Pope (2011) explore in their study?
The effect of being behind at halftime in basketball
Investigates whether losing can lead to winning
Fill in the blank: Each hamlet is not simply either a complier or not; there is a continuum of _______.
compliance
Discusses how compliance varies in response to bombing intensity in different hamlets
What type of average treatment effect must be considered for hamlets according to the text?
Weighted average treatment effect
Each hamlet is weighted based on the extent to which bombing responded to the score
