Experimental Causal Designs

Question 1

What is Experimental Causal Designs?

Answer 1

Experimental causal designs involve comparing counterfactual states of the world and explicitly assigning units to treatment conditions to estimate causal effects.

Question 2

What is Causal Inference?

Answer 2

• A causal inquiry involves comparing counterfactual states of the world.
• A data strategy is experimental when units are explicitly assigned to treatment conditions.

Question 3

What is a Two Arm Randomised Experiment?

Answer 3

• Subjects are randomly assigned to one of two conditions (treatment/control or potentially two different treatments).
• Each unit has two potential outcomes: treated and untreated.

Question 4

What are Cluster Randomised Experiments?

Answer 4

Entire groups (e.g., classrooms, localities, households) are assigned to treatment or control conditions.

Question 5

What are some challenges to Cluster Randomised Experiments?

Answer 5

Higher Variance

Clustering is generally less desirable statistically (higher variance) but may be necessary

Bias from Uneven Cluster Sizes

Question 6

What are Subgroup Designs?

Answer 6

Understand differences in treatment effects between subgroups. It is not randomised. Difference in CATEs.

Question 7

What are Factorial Experiments?

Answer 7

Experiments with multiple treatments (e.g., Treatment 1 with 2 levels, Treatment 2 with 2 levels).

Analyze how the effect of one treatment depends on the level of another treatment.Interaction effects can reveal complex relationships between treatments.

Question 8

Advantages of Factorial Experiments?

Answer 8

• Efficiency:Address multiple research questions simultaneously,potentially using a smaller total sample size compared to running separate experiments for each question.
• Rich Information:Explore how different treatments interact and influence each other’s effects (e.g.,how fertilizer A affects plant growth depends on whether irrigation is used).

Question 9

What are Placebo Controlled Experiments?

Answer 9

A placebo-controlled experiment involves comparing a treatment group receiving the actual treatment with a control group receiving a placebo, typically used to assess the effectiveness of a treatment by ensuring any observed effects are not simply due to the placebo effect.

Question 10

What are some advantages and disadvantages to placebo controlled experiments?

Answer 10

Benefit:

• Can increase precision over encouragement designs, particularly when the underlying compliance rate is low.

Drawback:

• Over/under sampling of placebo and treatment groups can lead to increased variance and reduced power, similar to encouragement designs.

Question 11

What are Stepped Wedge Experiments?

Answer 11

• Treatment is rolled out sequentially over multiple time periods to groups of units (e.g., schools, hospitals).
• Sometimes referred to as a “waitlist design.”

Question 12

What are some advantages or disadvantages?

Answer 12

Strengths:

• More ethical than withholding treatment from a group entirely.
• May have more power for the same number of total units compared to a two-arm randomized experiment (due to sequential rollout and observing units over time).

Weaknesses:

• More expensive than a two-arm randomized experiment for the same number of units.
• Not ideal when time effects impact different units differently (heterogeneity in time effects).

Question 13

What are Randomised Saturation Experiments?

Answer 13

Multi-level design with cluster randomization, used when the treatment effect depends on the treatment status of other units within a hierarchical structure (e.g., vaccination rates within a neighborhood). Analyzes how the saturation level (proportion of treated units within a cluster) influences treatment effects.