Lecture 5

Question 1

What does the predictive updating quality do?

Answer 1

Quantifies how well each of the values predicted the observed effect, >1 means support for your current data (prior beliefs) and <1 is a “surprising” result

Question 2

Florida’s bob figure has 5 letters, name what each letter indicates on the figure

Answer 2

A. Probability that Bob’s IQ is under 70 based on prior beliefs
B. Same prob., but based on posterior beliefs
C. Most probable singular value on the posterior belief
D. Number that indicates that an IQ of 75 is 1.5 times more likely than 70

Question 3

What are two rules in binomial and what are you looking at?

Answer 3

1. Data belongs to one of two discrete categories
2. The data sequence is exchangeable (order is irrelevant)

> You’re looking at the chance of encountering one of the categories in any one trial

Question 4

How do a and b values affect the beta distribution?

Answer 4

a = successes and b = failures, they represent, in the prior distribution, which values you think more or less likely. When collecting data for the binomial you encounter either successes or failures, each data point either provides evidence against or for your prior distribution.

Question 5

What does a beta distribution of a=1 and b=1 look like, a=100, b=100?

Answer 5

a=1, b=1 means that you have observed one success and one failure, as such, each value is equally likely at 1

for 100 on both, it is clear that there is a mostly equal chance for both categories to be encountered in any trial, as such a normal distribution around .5 is seen

Question 6

beta distribution of a=1 and b=0?

Answer 6

There is more evidence pointing towards trial a being likely, if the prior was an equal distribution, this will be a linear line with the highest point at 1

Question 7

What can be seen in the case of a=13 and b=8 (with a=1 and b=1 as prior)

Answer 7

A normal distribution that is moved towards the right, the highest point being around .65

Question 8

What is underfitting?

Answer 8

When a model is too simple to capture the data, meaning it will not be a good predictor and replicable structures are taken as noise

Question 9

What is overfitting?

Answer 9

When the model is too complex for the data, it will not be a good predictor and it will take noise and make it a replicable structure

Question 10

Occam’s razor?

Answer 10

Answers that require the least amount of assumptions are usually best

Question 11

What’s the deal with risky predictions?

Answer 11

They gain a large boost in credibility when evidence is found for it, but are also very easily discredited if the case was the other way around

See this as betting on one horse (aka lose or win all) vs. betting a little bit on all horses

Question 12

The risk reward relationship is established, why is this not the case for complex models?

Answer 12

Complex models have to predict too many things, (again horse betting)

Question 13

What is the bayes factor? (words)

Answer 13

Ratio of predictive performance

Question 14

What does bayes factor do opposed to opposed to the p value?

Answer 14

p only looks at the H0, so “one side of the coin”, you either reject or you keep it. Bayes compares two hypothesis and sees under which the data is most likely (H0 vs H1)

Question 15

What’s BF10 and B01 and what colours in the pizza pie indicate H1 and H0?

Answer 15

BF10 = H1 (red in pizza pie) and BF01 = H0 (white in pizza pie)

Question 16

What does evidence of absence mean and what does absence of evidence mean?

Answer 16

evidence of absence = the data is providing more evidence to count the H0 more likely

absence of evidence = there is no evidence leaning either way, the data isn’t likely under either hypothesis

Question 17

What are three advantages of Bayes factor (NOT evidence of absence shit)?

Answer 17

1. Quantifies evidence instead of forcing an all or none-decision
2. Allows evidence to be monitored as data comes in
3. Applies to data from the real world for which no sampling can be articulated (idk wtf this point means help)

Question 18

There is a large pitfall with bayes factor, called the bayes rate fallacy, what is this?

Answer 18

Interpretation of bayes is wrong. BF01 = 3 means that the data is 3 times more likely under H0 than H1, however people interpret it as H0 being 3 times more likely than H1

Question 19

What is the bayes factor dependent on, besides yakow, data?

Answer 19

BF is dependent on the question asked (aka changing the likelihood to be more precise or to distribute bets over many likelihoods) and the prior distribution you use

Question 20

Bayes factor can range a lot, what does BF10 = 1 mean?

Answer 20

Absence of evidence, data is equally likely under either hypothesis

Question 21

Bayes factor can range a lot, what does BF01 = 13 mean?

Answer 21

There is strong evidence pointing towards the data being more likely under the H0

Question 22

Bayes factor can range a lot, what does BF10 = 6 mean?

Answer 22

There is moderate evidence pointing towards the data being more likely under the H1

Question 23

Bayes factor can range a lot, what does BF01 = 47 mean?

Answer 23

There is extremely strong evidence for the data being more likely under the H0

Question 24

Bayes factor can range a lot, what does BF10 = 2 mean?

Answer 24

Anecdotal evidence for data being more likely under the H1

Question 25

Bayes factor can range a lot, what does BF01 = 167 mean?

Answer 25

Extreme evidence for data being more likely under H0