Lecture 2: Causal inference with experimental method Flashcards

1
Q

What is an experiment?

A
  • Golden standard
  • Includes intervention/treatment
  • Keep everything constant except for the treatment itself
  • Clean design? -> simple t-test/ANOVA
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Random Assignment

A
  • Equal chance
  • Characteristics of participants between two groups do not differ (limit confounding variables)
  • Small but a sufficiently large sample
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Simple randomization

A
  • Toss a coin (random)

- Limitation -> not equal number of persons between groups

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Block randomization

A
  • Randomize subjects into groups
  • Sample size is relatively small
  • Not intended for controlling confounding variable
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Stratified randomization

A
  • Limit the undesired influence of a known factor
  • Based on more than one factor
  • Number of strata should be in proportion with the target sample size
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Laboratory Experiment

A
  • Control setting
  • Participants informed and compensation
  • Allows intrusive design
  • Reduces chance of confounding variables
  • Improves internal validity but reduces external validity
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

When should we use laboratory experiment?

A
  • Stake of making a mistake is high
  • Confounding variables tend to occur
  • Setup is not feasible in real-world
  • Budget and participants available are limited
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Field experiment

A
  • Impact of a treatment in the real-world
  • Enabled by digitalization
  • Less control over other confounding factors
  • Spillover and crossover effects
  • Ethical issues
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Application in marketing

A
  • Academic -> testing a theory
  • Company -> testing an intervention)
  • Public policy -> testing an intervention
  • Litigator -> testing a hypothesis based on intuition
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Designing an experiment

A
Step 1: Formulate your hypothesis
Step 2: Select an appropriate sample
Step 3: Set up and run your experiment
Step 4: Analyse your data & interpret your findings
Step 5: Reformulate your hypothesis
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Research Question

A

A general statement of your observation or assumption

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

A scientific hypothesis should be

A
  1. Testable
  2. Falsifiable
  3. Stated as an expected relationship between variables
  4. Simple and easy to understand
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Practical concerns sample size

A
  • Availability participants
  • Costs
  • Time management
  • Data collection method
  • Ethical concerns
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Power analysis

A

Assess minimum sample size required given a significance

level, effect size, and statistical power.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Power Analysis Advantages

A
  • Limit p-hacking
  • Avoid over (i.e., waste of resources) or under power (i.e., unable to detect
    an effect when there is one)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Type II error

A

Finding an effect when there is no effect to be found

17
Q

Type I error

A

Falsely rejecting the null hypothesis

18
Q

Set up and run your experiment

A
  • Research question
  • Resource available
  • Access to participants
  • Type of experiment
  • Design: between/ within subjects design
  • Experimental groups and intervention/ treatment
  • Pilot test & manipulation check
  • Dependent variables
  • Threats to classical assumptions
19
Q

Other considerations

A
  • Demand characteristics
  • Motivation
  • Language issues
  • Gender and physical appearance
  • Control variables
  • Internal factor
  • External factors
  • Ethical concerns
20
Q

Paired sample t-test

A

Compare the means of the same group across two

time points

21
Q

Independent sample t-test

A

Compare the means of two independent groups

22
Q

Analysis of variance (ANOVA)

A

Compare the means of more than two groups

23
Q

ANCOVA

A

ANOVA with control variables

24
Q

Chi-square test

A

Categorical variables

25
Q

Interpret your findings with caution

A
  • Explore potential confounder (seasonal change)
  • Take context into account
  • Mine your data (with caution)
  • Follow up tests (Robustness check & Replication)
26
Q

Limitations of experimental method

A
  • High control and internal validity but low ease of use
  • Infeasibility of randomized experiments → Ethical concerns
  • Difficult to implement
  • High financial and opportunity cost
  • Availability of participants