Observational Causal Design

Question 1

What is an Observational Causal Design?

Answer 1

An observational causal design studies how the world naturally assigns conditions (treatments) and analyzes their effects, but can’t definitively prove cause and effect.

Question 2

What is Process Tracing?

Answer 2

Process tracing is a qualitative method used to understand how a treatment (X) might cause an outcome (Y) within a single case or across multiple cases.

It relies on a pre-existing theoretical model that outlines the potential causal pathway from the treatment to the outcome.

Question 3

What techniques are used to infer conclusions from process tracing?

Answer 3

1. Hoop test: Observing a mediator (M) can be a positive sign, but doesn’t definitively prove X causes Y. (Like seeing a hoop doesn’t guarantee a successful jump)
2. Smoking gun test: Observing a moderator (W) that influences both M and Y provides stronger evidence for the causal pathway. (Like a puff of smoke directly linking a gun to the shooter)
3. DAG (Directed Acyclic Graph) used to illustrate causal model and specify restrictions on causal relations.

Question 4

What is Multivariate Regression (or selection on observables)?

Answer 4

Multivariate regression is a technique used in observational causal design to control for confounding variables in a regression model to address selection bias.

Question 5

What does the validity of your causal claims depend on?

Answer 5

The validity of causal claims rests on the accuracy of your model regarding backdoor pathways

Question 6

Why do you adjust for a confounder?

Answer 6

Adjusting for a confounder (X) in a model with cause (D) and outcome (Y) removes confounding bias (YES).

Question 7

Why do you adjust for a moderator?

Answer 7

Adjusting for a moderator (X) does not introduce bias and may improve estimate precision (YES).

Question 8

Why do you adjust for a collider?

Answer 8

Adjusting for a collider (X) introduces collider bias (NO).

Question 9

Why do you adjust for a mediator?

Answer 9

Controlling for a mediator (X) estimates the “direct effect” of D on Y, while not controlling gives the “total effect” (MAYBE).

Question 10

What can be used to adjust for a confounder if it biases the treatment outcome relationship?

Answer 10

Selection on Observables Matching: This creates comparable “treatment” and “control” groups based on the confounder (X). It adjusts the covariate properties of these groups to reduce selection bias.

Question 11

What is Difference in Differences (DID)

Answer 11

Difference-in-differences methods involve comparing changes over time between a group that receives a treatment and a group that does not, aiming to isolate the treatment effect by subtracting out unit-specific characteristics and trends affecting all units.

Question 12

What steps are their in the DiD method?

Answer 12

1. Take “pre-treatment” and “post-treatment” measurements of the outcome variable (Y) for both the treated unit and a comparator unit.
2. Calculate the difference in the outcome variable for each unit between pre- and post-treatment periods.
3. Compare the differences-in-differences between the treated and comparator groups.

Question 13

What assumptions does DiD make?

Answer 13

1. No Time-Variant Unobservables: No unit-specific factors changing over time, besides the treatment, influence the outcome.
2. No Heterogeneous Temporal Effects: The impact of time is the same for both the treated and comparator units.
3. Parallel Trends: In the absence of treatment, both groups would have followed similar outcome trajectories. (Unobservable counterfactuals)

Question 14

What is Propensity Score Matching?

Answer 14

This is an alternative to matching when there are multiple confounders. However, it doesn’t address unobserved confounders.

Question 15

What are some limitations to DiD?

Answer 15

The parallel trends assumption can be violated, leading to biases. Diagnostics involve examining pre-treatment trends to assess this assumption.

Question 16

What is the Instrumental Variables Technique?

Answer 16

Instrumental Variables (IV) is a powerful technique used in research to estimate causal effects when confounding variables or challenges with other methods exist. It works in a two-stage process to address omitted variable bias, measurement error, and simultaneity issues.

Question 17

What do IV address?

Answer 17

IVs address confounding concerns, especially when controlling for all confounding variables is difficult or assumptions in other designs (like parallel trends) are unlikely.

Question 18

What are Instrumental Variables and what is their notation?

Answer 18

Z: Variables that influence the treatment variable (D) but not directly affect the outcome variable (Y) except through D (“as-if randomly assigned”).

• Are not correlated with unobserved confounders (exogeneity).
• Have a statistically significant effect on the treatment (non-zero effect).

Question 19

What is the advantage to the IV method?

Answer 19

IVs offer a way to estimate causal effects without concerns about confounding.

Question 20

IV is a two stage process, outline these processes:

Answer 20

1. Stage 1: Identify Effect of Z on D:
   
   • Regress D on Z to estimate the exogenous part of D influenced by Z. (We lose some information about D due to this estimation.)
2. Stage 2: Identify Effect of D on Y:
   
   • Use the estimated D from stage 1 as an independent variable in a separate regression to estimate the causal effect of D on Y.

Question 21

What are some conditions for Validity for the IV method?

Answer 21

1. Relevance: Z must affect the D
2. Exogeneity: there must be no confounding between Z and Y
3. Exclusion Restriction/ No Direct Effect: Z should only affect Y through D
4. Monotonicity: The effect of Z on D should be 0 or positive for all units.

Question 22

What is LATE in relation to IV?

Answer 22

When treatment effects vary across individuals, IV estimates the Local Average Treatment Effect (LATE)

Question 23

What are the four populations with heterogenous treatment effects?

Answer 23

1. Compliers: Treatment is affected by the instrument in the “right” direction (e.g., influenced by a lottery to participate in a program).
2. Defiers:Treatment is affected by the instrument in the “wrong” direction (e.g., avoid a program due to a lottery prompting participation).
3. Never-Takers: Never take treatment, regardless of the instrument.
4. Always-Takers: Always take treatment, regardless of the instrument.

Question 24

What is RDD?

Answer 24

Regression Discontinuity Design is a quasi-experimental design that leverages a sharp cutoff point in a variable (running variable) to create comparable treatment and control groups.

This helps overcome selection bias, a common issue in observational studies.

Question 25

What characteristic defines treatment assignment in a RDD

Answer 25

In RDD, treatment is assigned based on whether a unit’s characteristic (running variable) crosses a specific threshold value.

Question 26

What is the main limitation of RDD?

Answer 26

As we get closer to the cutoff point, there’s less data available, making analysis challenging. Limited generalizability can also be a concern.

Question 27

What is the LATE estimated by RDD?

Answer 27

RDD estimates the treatment effect for units closest to the cutoff, where they are most comparable.

Question 28

What is the role of running variable in RDD?

Answer 28

The running variable is the characteristic that determines treatment assignment based on the cutoff point.

Question 29

What are some considerations for RDD designs?

Answer 29

Bandwidth Selection: Choosing the appropriate window size around the cutoff point to balance bias and variance.

Covariate Balance: Checking for imbalances in observable characteristics between the two groups.

Diagnosis: Assessing potential threats to the validity of the RDD design.