Experiments Flashcards

1
Q

What is “control’?

A

Being able to manipulate one behavioural factor of interest while holding all other factors constant

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What are the three methods for constructing a counterfactual?

A

Difference in differences
Propensity matching
Laboratory experiments

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What assumptions are required to hold for the difference in differences model to provide a valid counterfactual?

A

Parallel trends assumption, observed and unobserved characteristics remain constant over time.
Groups respond to treatment the same way.
One group cannot be exposed in isolation to a third factor.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is propensity matching?

A

Subjects are matched by the probability of being treated using a profit estimator. Treated are compared to untreated, with equal probability scores.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What assumptions are required for propensity matching to produce a valid counterfactual?

A

Treatment and control groups are the same in absence of the treatment.
Treatment and control groups respond in the same way to the treatment.
Neither group can be exposed in isolation to a third factor.
Unobserved characteristics are the same for treated and untreated.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What assumptions must be met for a laboratory experiment to produce a valid counterfactual?

A

Treatment and control groups must be the same in absence of the treatment, on average.
Groups should respond to treatment in the same way.
Groups cannot be exposed in isolation to a third factor.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What does control solve for?

A

Confounds

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What does randomisation solve for?

A

Selection bias.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What 3 qualities mean that experiments allow causal inference?

A

The counterfactual can be observed.
Control means there are no confounds.
Randomisation eliminates selection bias.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What 5 precepts achieve control?

A
  1. Non-satiation.
  2. Salience.
  3. Dominance.
  4. Privacy
  5. Parallelism
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Describe non-satiation

A

Given a costless choice, more of the reward medium is always preferred to less.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Describe salience

A

The reward structure matches performance. Better performance leads to greater rewards. These promised rewards are honoured by the experimenter.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Describe dominance

A

The reward structure outweighs and subjective costs of participating in the experiment

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Describe privacy

A

Subjects are only given information regarding their own payoffs, unless explicitly part of the hypothesis being tested.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Describe parallelism

A

Subjects and institutions in the lab are expected to behave similarly outside the lab, given similar ceteris paribus conditions.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What two precepts satisfy an Economic experiment?

A

Non-satiation and salience

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Which two precepts are required for an economic experiment to be controlled?

A

Dominance and privacy.

18
Q

What 3 levels can independent observations be made at?

A

Individual level, group level, session level.

19
Q

What is the strategy method in experimental design?

A

Subjects list their choice for every possible outcome of the game.

20
Q

Why is incentivising/paying beliefs potentially a bad idea?

A

Hedging/risk-pooling. Subjects may guarantee some payoff by behaving one way but listing beliefs differently in order to guarantee some level of payment.

21
Q

Describe the intuition of the Labor Framework in incentive design

A

Incentives induce higher effort and therefore more optimal performance.

22
Q

What is the flat maximum critique?

A

If incentives are too insensitive around the optimum performance, this may fail to induce effort or reduce effort variance in subjects.

23
Q

Describe the intuition of the Capital Labor Production framework

A

Capital is the variety of approaches to solving an experimental task, e.g. skills, experience. Subjects bring this.
Production requirements are the kinds of capital necessary for good performance.
To induce effort, capital needs to suit production.

24
Q

What are the main consequences of the Capital Labor Production framework?

A

Capital can have as strong an effect on performance as incentives.
Capital should be suited to production.
Nature of the task matters, can be too easy or hard.
Experimental design can influence production, e.g. fun task.
Capital can be depleted.

25
Q

What are the 4 strategies for measuring utility?

A
  1. Direct scaling
  2. Certainty equivalence
  3. Probablity equivalence
  4. Multiple choice/price lists
26
Q

Describe the direct scaling method

A

Subjects rank items or score them on an ordinal scale of value

27
Q

What are the benefits of the direct scaling method?

A

Simple to understand for subjects

28
Q

What are the downsides of the direct scaling method?

A

Produces only ordinal data.

Cannot elicit utility functions.

29
Q

Describe the certainty equivalence method

A

Subjects provide the fixed amount that would make them indifferent with a lottery

30
Q

What are the benefits of the CE method?

A

Can elicit utility functions.

31
Q

What are the issues with the CE method?

A

Requires assumption of Expected Utility preferences.

32
Q

Describe the probability equivalence method

A

Subjects give the probability of a lottery with fixed payoffs that makes them indifferent to a fixed certain amount.

33
Q

In what situations is the PE method most useful?

A

When outcomes are non-continuous, e.g. states of being

34
Q

What are the benefits of the PE method?

A

Can be used in more situations.

Can elicit utility functions.

35
Q

What are the downsides of the PE method?

A

Requires assumption of Expected Utility preferences.

Can be hard to understand.

36
Q

What is incentive compatibility?

A

Incentives should induce honest responses or behaviour

37
Q

What mechanism makes CE/PE incentive compatible?

A

The BDM mechanism

38
Q

Give an example of the BDM mechanism

A

Subjects are shown an item.
Subjects state willingness to pay.
A random number is generated. If lower than the stated price, subject buys the item. If higher, they don’t buy it.

39
Q

Why does the BDM mechanism make the CE/PE methods incentive compatible?

A

It is always the dominant strategy for the subject to provide their honest value.

40
Q

Describe the multiple price/choice list method

A

Subjects choose between two lotteries

41
Q

What are the benefits of the MPL method?

A

Easy to understand.

Subjects don’t have to report absolute values.

42
Q

What are the downsides of the MPL method?

A

MPL estimates intervals of utilities, not point estimates.
Subjects can be inconsistent t if not given boundaries.
There can be framing effects.
Incentives may be an issue, e.g. wealth effects.