5. Market Experiments, Lab vs Field

Question 1

Smith 1962 model assumptions

Answer 1

Homogeneous good
Large number of buyers and sellers
Price taking behaviour

Question 2

Describe the set up of Smith 1962

Answer 2

-subjects randomly assigned as buyer or seller
-buyers know value and sellers know cost
-profit= price- value
-buyers and sellers can post and accept prices at any time during a trading session
-no incentives used

Question 3

What does smith say about the behaviour of markets?

Answer 3

“All the supply and demand schedules can do is set broad limits on the behaviour of the market” (Smith 1991)

Question 4

Results of Smith 1962

Answer 4

-Prices converge quickly to competitive equilibrium prediction.
-quick adjustment to new equilibrium after shifts in supply/demand
-convergence is more erratic for steeper supply and demand schedules
-distribution of rents affects convergence to equilibrium- convergence from below if sellers’ rent is relatively higher
-buyers rent relatively higher after demand shock, convergence from above
-prices slightly above the equilibrium price for perfectly elastic supply curve

Question 5

Describe the set up of the Gode & Sunder 1993 study

Answer 5

Computer simulation of zero intelligence traders
2 kinds of traders
1. No budget constraint: ZI buyers/sellers randomly bid/ask prices
2. With budget constraint: ZI buyers/sellers bid/offer randomly below/above their value/costs
Transaction whenever a bid price is at least as high as an ask price

Question 6

Results of Gode & Sunder 1993

Answer 6

High efficiency with budget constraints
Convergence to equilibrium price

Question 7

What are posted offer markets

Answer 7

When one side of the market makes a price offer and publicly announced it as a take it or leave it. E.g. supermarket

Question 8

How do posted offer markets differ in prices compared to double auction markets?

Answer 8

In posted offer markets transaction prices tend to be higher if sellers post prices and lower if buyers post bids

Question 9

Natural field experiment

Answer 9

Where the experimenter conducts an experiment in a naturally occurring environment where participants don’t know that they are taking part in an experiment

Question 10

Describe set up of Gachter, Orzen, Renner & Starmer 2009

Answer 10

Email reminding delegates to register by early registration deadline
Two frames
-penalty of £50 for registering late
-discount of £50 for registering on time

Question 11

Results of Gachter, Orzen, Renner & Starmer 2009

Answer 11

Significantly higher number of junior researchers meet deadline with penalty frame (93% vs 67%) no difference in senior researchers

Question 12

Advantages of lab experiments compared to field (happenstance) data

Answer 12

-experiments can be designed to answer questions that can’t be answered with field data
-natural experiments rarely occur
-tight control of experimental environment allows ceteris paribus comparison between treatments
-experimenter can control otherwise unobservable variables

Question 13

How does lab experiment differ from outside world

Answer 13

-presentation of the task in lab is artificial
-subject pool (usually students), potentially self selection bias
-subjects are inexperienced
-size of stakes

Question 14

What is a common scientific argument about external validity in a lab setting?

Answer 14

External validity is not of first rate importance if experiment is designed to test a theory

Question 15

What is the policy view of external validity in a lab setting?

Answer 15

Generalisability is crucial since we want to extrapolate from lab to (specific) field setting

Question 16

According to Harrison & List 2004, what constitutes a field experiment?

Answer 16

-subject pool
-info that the subjects bring to the task
-commodity
-task or trading rules applied
-stakes
-environment that the subject operates in

Question 17

What do Falk & Heckman 2009 argue about lab vs field experiments?

Answer 17

It is not obvious whether the lab x* or field x** is more informative for the third condition unless a more tightly specified model is postulated or a precisely formulated policy problem is specified.
The general quest for running experiments in the field to obtain more realistic data is misguided. In fact, the key issue is what is the best way to isolate the effect of X1 whilst holding constant (X2,…,Xn)

Question 18

What does scrutiny refer to?

Answer 18

The fact participants are aware they are being observed in the lab

Question 19

Describe set up of Erik Snowberg & Leeat Yariv (2021)

Answer 19

Motivation to address concerns about external validity of experiments with student participants. Large scale incentivised survey with Caltech uni students, representative US sample, Amazon mechanical Turk subjects. They elicit risk aversion, altruism, over confidence, over precision, attitudes towards gender and race, various strategic interactions

Question 20

Results of Erik Snowberg & Leeat Yariv (2021)

Answer 20

-behaviours are similar across samples
-non student samples exhibit higher levels of noise
-additional set of lab experiments show no evidence of observer effects

Question 21

Methodological issues

Answer 21

-documentation of experimental design, procedures and instructions, transparency
-replication
-interpretation of results
-single and double blind procedures
-ethics and integrity

Question 22

Measure for convergence in Smith 1962

Answer 22

Alpha = SD of exchange price/ equilibrium price

Question 23

4 versions of experiment according to Harrison & List 2004

Answer 23

-conventional lab experiment: standard subject pool, abstract framing and imposed set of rules
-artefactual field experiment: same as conventional but not students
-framed field experiment: same as artefactual but with field context in either commodity, task or info set
-natural field experiment: same as framed field but subjects don’t know they are in an experiment

Question 24

3 keys features of lab and field experiments

Answer 24

Control
Replicability
Scrutiny

Question 25

Which common objections against lab experiments can be addressed with more lab experiments?

Answer 25

-participant pool
-experience/inexperience
-stake size
-context
-scrutiny

Question 26

Argument by Kessler & Vesterlund 2015

Answer 26

The focus on quantitative external validity is misplaced for many experimental studies as the emphasis in lab studies is to identify the direction rather than the precise magnitude of an effect.

Question 27

What should be the general takeaway of field vs lab experiments?

Answer 27

-empirical research in economics can use different sources of data- happenstance, lab, field experiments, surveys
-different sources of data can complement each other and have different comparative advantages
-varieties of observations and series of experiments can help improve our state of knowledge

Question 28

Falk & Heckmans retort to claims against lab experiments

Answer 28

1. Low stakes- there is no obvious level at which stakes are in real life.
2. Small sample sizes- effective methods have been developed for analysing small sample sizes
3. Hawthorne effects- being observed isn’t an exclusive feature of lab experiments. Can use double blind.
4. Self selection- Selection is a feature of both lab and field experiments.