Reasoning about probabilities

Question 1

What is debated amongst mathematicians regarding the nature of probability?

Answer 1

Whether it is Bayesian or Frequentist.

Question 2

How do Bayesians define probability?

Answer 2

Probability refers to a subjective degree of confidence, and as one can express confidence that a single event will occur, one can express the probability of a single event.

Question 3

How do Frequentists define probability?

Answer 3

Probability is always defined over a reference class such as infinite number of coin tosses. Single events don’t belong to a reference class so they cannot have a relative frequency or probability.

Question 4

What do psychologists mean when they refer to normative probabilities?

Answer 4

If the probability is normative, the output of the system is the same that would be returned by a Bayesian machine. This is not related to whether the internal process is Bayesian.

Question 5

What has much psychological research been conducted with reference to?

Answer 5

Single event probabilities (posterior probability).

Question 6

What is posterior probability?

Answer 6

The conditional probability that is assigned after the relevant evidence is taken into account; in psychology the probability of a hypothesis (H) given data (D) = p(H|D).

Question 7

What did Kahneman and Tversky (1972) write about humans and Bayesian evaluation of evidence (probability calculating)?

Answer 7

Humans are “not Bayesian at all”.

Question 8

What did Gould (1992) state about the human mind and probability?

Answer 8

“…our minds are not built…to work by the rules of probability”

Question 9

What can the heuristics and biases literature be used to demonstrate about probabilistic reasoning?

Answer 9

Essentially, humans are irrational when it comes to probabilistic reasoning.

Question 10

Can we reason according to Bayes Theorem according to evolutionary psychologists?

Answer 10

Yes, as long as the information is presented in a format which we have evolved to process (i.e. not probabilities, a modern form of mathematical notation).

Question 11

What is Kahneman and Tversky’s view on why humans are irrational when it comes to probabilistic reasoning?

Answer 11

Bayesian reasoning is too complex so we use heuristics, which are necessary because of the poverty of input. The mechanism and/or task are too complex and we’re limited by our inability to reason.

Question 12

What is Cosmides and Tooby’s view on why humans are irrational when it comes to probabilistic reasoning?

Answer 12

We cannot do probabilistic reasoning because the information is in the wrong format (not frequency). The mechanism needed is simple (a few lines of computer code), and the task is therefore no more complex.

Question 13

What is the problem with Cosmides and Tooby’s argument regarding probabilistic reasoning?

Answer 13

It is a tautology and gives no evolutionary reason why Bayesian reasoning shouldn’t have developed - even simple sea slugs exhibit habituation and all vertebrates can be classically conditioned. These processes can be described as Bayesian inferences - animal learning approximates Bayes Theorem.

Question 14

What processes can be described as Bayesian inferences?

Answer 14

Habituation and classical conditioning - learning approximates Bayes Theorem.

Question 15

What are the advantages of the frequentist format?

Answer 15

• The number of events (sample size), which indicates the reliability of the decision, is retained
• Permits easy updating as new information is collected
• Reference classes can be constructed post-hoc according to new information as the reference class changes

Question 16

What did Tversky and Kahneman (1982) study?

Answer 16

Base rate neglect in the city cabs experiment.

Question 17

Describe Tversky and Kahneman (1982)’s experiment.

Answer 17

Participants are told that a cab was involved in a hit and run accident in a place where 85% of the cabs are green and 15% are blue. A witness identified the cab involved as blue, and the court found their reliability to be 80% correct. Participants were asked: “what is the probability that the cab involved was blue not green?”

Question 18

What did Tversky and Kahneman (1982) find?

Answer 18

The actual probability that the taxi was blue = 0.12/(0.12+0.17) = 0.41, however most participants think the taxi was more likely to be blue e.g. >0.50 and most stated p=0.80.

Question 19

What did Tversky and Kahneman (1982) attribute base rate neglect to?

Answer 19

The representativeness heuristic - participants focus on the witness’ accuracy and neglect the base rate of the city’s cabs.

Question 20

What did Casscells et al. (1978) study?

Answer 20

Base rate neglect using the medical diagnosis problem.

Question 21

What did Casscells et al. (1978) do?

Answer 21

Asked medical students: If a test to detect a disease whose prevalence is 1/1000 has a false positive rate of 5%, what is the chance that a person found to have a positive result actually has the disease? Assuming that you know nothing about the person’s symptoms or signs.

Question 22

What did Casscells et al. (1978) find?

Answer 22

18% responded 2% (correct Bayesian inference)

45% responded 95% (neglected base rate)

Question 23

What did Casscells et al. (1978) conclude?

Answer 23

That even medical students ignore base rates for diagnostic problems.

Question 24

What did Cosmides & Tooby (1996) do?

Answer 24

Presented a similar medical diagnosis problem to Casscells et al. (1978) in both frequency and probability formats. Participants had to estimate how many of those out of 1000 randomly selected who tested positive actually have the disease.

Question 25

What did Cosmides & Tooby (1996) find?

Answer 25

When presented in a frequency format, around 95% of participants correctly answered 2%.

Question 26

What did Cosmides & Tooby (1996) conclude?

Answer 26

That frequency format abolishes base rate neglect. The relevant modules cannot reason about probabilities because it’s domain specific (can only be accepted in a frequency format).

Question 27

What is a problem with Cosmides & Tooby (1996)?

Answer 27

Griffin & Beuhler (1999) and Evans et al. (2000) couldn’t replicate their results.

Question 28

How is the relationship between base rate neglect and frequency format shown in the conjunction fallacy?

Answer 28

A single-event version is used - participants are asked to “rank with respect to their probability”, or “to how many out of a 100 who are like Linda do the following statements apply?”, with the usual Linda bank teller problem. The fact that the bank teller category includes feminists and non-feminists is clearer in the frequency format. Research has shown that participants are more likely to avoid the conjunction fallacy when it’s presented in a frequency format.

Question 29

How are base rate neglect and frequency format related to the Monty Hall problem?

Answer 29

Both Aaron & Spivey (1998) and Krauss & Wang (2003) found that frequency formats elicited higher switch rates than probability formats.

Question 30

What are preference reversals?

Answer 30

A key phenomenon that violates rational choice theory.

Question 31

Describe a typical preference reversal experiment, e.g. Lichtenstein & Slovic (1971).

Answer 31

Participants are presented with pairs of monetary gambles or ‘bets’ which contain the possibility of winning/losing certain amounts.
- One bet in each pair is relatively safe and has a high possibility of winning a small amount (P-bet)
- The other bet is more risky and has a small possibility of winning a large amount ($-bet)
Later the bets are presented individually and participants are asked to provide monetary value for each one.

Question 32

Why do preference reversals violate RCT?

Answer 32

Because typically participants prefer the P-bet in the choice phase but rate the $-bet with a higher monetary value, which according to Angner (2002) reflects its utility. Such inconsistent behaviour appears to be a robust phenomenon and suggests human irrationality is ‘systematic and widespread’.

Question 33

What did Tunney (2006) discover?

Answer 33

That preference reversals are diminished when presented as frequencies.

Question 34

Describe the mammography problem (probability format).

Answer 34

The probability of breast cancer is 1% for a woman aged 40 who participates in routine screening, and if a woman has breast cancer, there’s an 80% probability she’ll have a positive mammogram (false pos. 9.6%). A woman of this age group has a positive mammogram in routine screen, what’s the probability she’ll have breast cancer? Actually 7.8%, but 95% of doctors say it’s 80%.

Question 35

Describe the mammography equivalent problem (frequency format).

Answer 35

Imagine a neutral healer in a primitive society (no probability theorem). In her lifetime she’s seen over 1000 people, 10 had the disease, 8 of which showed the symptom. Of the 990 not afflicted, 95 did. A new patient has the symptom, do they have the disease? (8/8+95).

Question 36

Why do frequencies elicit normative reasoning?

Answer 36

According to Gigerenzer & Hoffrage, Bayesian computations are simpler in frequency format as only the absolute frequencies of (D|H) and (D|¬H) need to be stored. Base rates aren’t needed so you get neglect when they are needed in probability formats. Frequencies may elicit normative reasoning because they are so similar to natural sampling.

Question 37

What three methods did Sedlemeier & Gigerenzer (2001) use to teach Bayesian reasoning?

Answer 37

In all, participants were given all the relevant information and told which numbers corresponded to which part of the formula.

1. Rule training
   - Given formula, all participants needed to do was the calculation to return p(H).
2. Frequency grid
   - Participants indicated p(H) by selecting one of two grids with the correct and an incorrect answer.
3. Frequency tree
   - The tree doesn’t represent individual cases but constructs a reference class (total number of observations) for each branch.

Question 38

What did Sedlemeier & Gigerenzer (2001) find?

Answer 38

Initially all groups showed improvement, but after a retention interval the rule training group’s performance declined, i.e. they began to neglect base rates again, whereas the frequency representation group retained their knowledge for a period of at least 5 weeks.

Question 39

What do Sedlemeier & Gigerenzer (2001)’s findings suggest?

Answer 39

That Bayesian computations are simpler when information is represented in natural frequencies.

Question 40

What did Goodie & Fantino (2001) show?

Answer 40

That base rate neglect disappears following exposure to outcomes but can return under time pressures.