Experiments Flashcards
What is “control’?
Being able to manipulate one behavioural factor of interest while holding all other factors constant
What are the three methods for constructing a counterfactual?
Difference in differences
Propensity matching
Laboratory experiments
What assumptions are required to hold for the difference in differences model to provide a valid counterfactual?
Parallel trends assumption, observed and unobserved characteristics remain constant over time.
Groups respond to treatment the same way.
One group cannot be exposed in isolation to a third factor.
What is propensity matching?
Subjects are matched by the probability of being treated using a profit estimator. Treated are compared to untreated, with equal probability scores.
What assumptions are required for propensity matching to produce a valid counterfactual?
Treatment and control groups are the same in absence of the treatment.
Treatment and control groups respond in the same way to the treatment.
Neither group can be exposed in isolation to a third factor.
Unobserved characteristics are the same for treated and untreated.
What assumptions must be met for a laboratory experiment to produce a valid counterfactual?
Treatment and control groups must be the same in absence of the treatment, on average.
Groups should respond to treatment in the same way.
Groups cannot be exposed in isolation to a third factor.
What does control solve for?
Confounds
What does randomisation solve for?
Selection bias.
What 3 qualities mean that experiments allow causal inference?
The counterfactual can be observed.
Control means there are no confounds.
Randomisation eliminates selection bias.
What 5 precepts achieve control?
- Non-satiation.
- Salience.
- Dominance.
- Privacy
- Parallelism
Describe non-satiation
Given a costless choice, more of the reward medium is always preferred to less.
Describe salience
The reward structure matches performance. Better performance leads to greater rewards. These promised rewards are honoured by the experimenter.
Describe dominance
The reward structure outweighs and subjective costs of participating in the experiment
Describe privacy
Subjects are only given information regarding their own payoffs, unless explicitly part of the hypothesis being tested.
Describe parallelism
Subjects and institutions in the lab are expected to behave similarly outside the lab, given similar ceteris paribus conditions.
What two precepts satisfy an Economic experiment?
Non-satiation and salience
Which two precepts are required for an economic experiment to be controlled?
Dominance and privacy.
What 3 levels can independent observations be made at?
Individual level, group level, session level.
What is the strategy method in experimental design?
Subjects list their choice for every possible outcome of the game.
Why is incentivising/paying beliefs potentially a bad idea?
Hedging/risk-pooling. Subjects may guarantee some payoff by behaving one way but listing beliefs differently in order to guarantee some level of payment.
Describe the intuition of the Labor Framework in incentive design
Incentives induce higher effort and therefore more optimal performance.
What is the flat maximum critique?
If incentives are too insensitive around the optimum performance, this may fail to induce effort or reduce effort variance in subjects.
Describe the intuition of the Capital Labor Production framework
Capital is the variety of approaches to solving an experimental task, e.g. skills, experience. Subjects bring this.
Production requirements are the kinds of capital necessary for good performance.
To induce effort, capital needs to suit production.
What are the main consequences of the Capital Labor Production framework?
Capital can have as strong an effect on performance as incentives.
Capital should be suited to production.
Nature of the task matters, can be too easy or hard.
Experimental design can influence production, e.g. fun task.
Capital can be depleted.
What are the 4 strategies for measuring utility?
- Direct scaling
- Certainty equivalence
- Probablity equivalence
- Multiple choice/price lists
Describe the direct scaling method
Subjects rank items or score them on an ordinal scale of value
What are the benefits of the direct scaling method?
Simple to understand for subjects
What are the downsides of the direct scaling method?
Produces only ordinal data.
Cannot elicit utility functions.
Describe the certainty equivalence method
Subjects provide the fixed amount that would make them indifferent with a lottery
What are the benefits of the CE method?
Can elicit utility functions.
What are the issues with the CE method?
Requires assumption of Expected Utility preferences.
Describe the probability equivalence method
Subjects give the probability of a lottery with fixed payoffs that makes them indifferent to a fixed certain amount.
In what situations is the PE method most useful?
When outcomes are non-continuous, e.g. states of being
What are the benefits of the PE method?
Can be used in more situations.
Can elicit utility functions.
What are the downsides of the PE method?
Requires assumption of Expected Utility preferences.
Can be hard to understand.
What is incentive compatibility?
Incentives should induce honest responses or behaviour
What mechanism makes CE/PE incentive compatible?
The BDM mechanism
Give an example of the BDM mechanism
Subjects are shown an item.
Subjects state willingness to pay.
A random number is generated. If lower than the stated price, subject buys the item. If higher, they don’t buy it.
Why does the BDM mechanism make the CE/PE methods incentive compatible?
It is always the dominant strategy for the subject to provide their honest value.
Describe the multiple price/choice list method
Subjects choose between two lotteries
What are the benefits of the MPL method?
Easy to understand.
Subjects don’t have to report absolute values.
What are the downsides of the MPL method?
MPL estimates intervals of utilities, not point estimates.
Subjects can be inconsistent t if not given boundaries.
There can be framing effects.
Incentives may be an issue, e.g. wealth effects.
