2nd mid term Flashcards
Can we observe both outcomes at the same time
No, it’s the fundamental problem of causal inference.
Reasons why experimental design is advantageous
-Time Order
-Other influences/factors can be excluded
-Random assignment prevents self-selection
Challenge of experimental design
external validity
How to address the causality issue in Observational studies
Using the experimental logic for observational studies and Quasi-experiments
How to address the causality issue in Experimental studies
- Using a variety of experimental designs, not just
classic experiments - Natural
Experiments - Lab Experiments
- Field
Experiments - Survey
Experiments
X 0 means
Post Test
X O O
Post test with control group: Compare two types of cases/groups of people
01 X 02
Pre and Post test: Compare a case to itself
01 02
X
01 02
Pre and post test design with a control group
X0 0
X3 0
X6 0
X9 0
Dosage Design: Compare cases of different manipulation strength
Most similar systems design
-Cases are as similar as possible
-Very common in regional studies
-Similarity of cases means we control
-If one factor is different between cases, and outcome is different, this is our probable cause for the outcome
-Selection of cases that take on similar values of confounding variables, but different values of a key independent variable
An example of most similar systems design
Why is Ghana more democratic than Togo
Most dissimilar system design
cases that differ on different values for independent variables but take on the same values for the dependant value which allows for the independent value to be disproven as a causal variable
Steps to test causality in observational studies
- Showing a correlation
- Excluding other factors
- Temporal Order
- Control group
- randomization of assignment or equivalent
- Need causal mechanism
Problems with the most similar design system
-The more complicated the operationalization the more difficult is the use of this approach
-Multiple causal factors and causal complexity are hard or impossible to determine.
-Cannot control for all variables
- Deterministic causality but reasoning should be probabilistic
-External validity is potentially low
What Natural experiment did john snow observe
Water contamination in london and association to cholera
Healy and Malhotra
having a sibling that is female and the ideological views of the subject natural experiment
Critique of natural experiments
-Might not always tackle the most important question
-data driven from theorizing
Threats to external validity for experimental designs
-Unrepresentative subjects or case
-Hawthorne effect
-Treatment is applied in a way we wouldn’t see in the real world
Hawthorne effect
People change their behavior because they are being observed (both internal and external).
Characteristics of Lab experiments
-Research in lab or controlled environment
-Researcher in full control
-Complete randomization into treatment and control groups
-Good for internal validity
-Often Cheaper, less time consuming and easier to replicate than field experiments
Lab experiment weaknesses
- Artificial environment - low realism.
- Demand characteristics - participants aware of experiment, may change behaviour to what they think is expected of them.
- Experimenter effects - bias when experimenter’s expectations affect behaviour.
Field experiments characteristics
- A field experiment takes place anywhere in a natural setting; it could take place in a school, hospital, the street or an office.
- People may behave more naturally than in laboratory - higher realism.
- Easier to generalise from results.
- People often do not know they are
being studied
Examples of Field experiments
Canvassing experiments Civic course experiments Vote Compass experiments Evaluating Programs/Policies
