Lecture 5 Flashcards
What does the predictive updating quality do?
Quantifies how well each of the values predicted the observed effect, >1 means support for your current data (prior beliefs) and <1 is a “surprising” result
Florida’s bob figure has 5 letters, name what each letter indicates on the figure
A. Probability that Bob’s IQ is under 70 based on prior beliefs
B. Same prob., but based on posterior beliefs
C. Most probable singular value on the posterior belief
D. Number that indicates that an IQ of 75 is 1.5 times more likely than 70
What are two rules in binomial and what are you looking at?
- Data belongs to one of two discrete categories
- The data sequence is exchangeable (order is irrelevant)
> You’re looking at the chance of encountering one of the categories in any one trial
How do a and b values affect the beta distribution?
a = successes and b = failures, they represent, in the prior distribution, which values you think more or less likely. When collecting data for the binomial you encounter either successes or failures, each data point either provides evidence against or for your prior distribution.
What does a beta distribution of a=1 and b=1 look like, a=100, b=100?
a=1, b=1 means that you have observed one success and one failure, as such, each value is equally likely at 1
for 100 on both, it is clear that there is a mostly equal chance for both categories to be encountered in any trial, as such a normal distribution around .5 is seen
beta distribution of a=1 and b=0?
There is more evidence pointing towards trial a being likely, if the prior was an equal distribution, this will be a linear line with the highest point at 1
What can be seen in the case of a=13 and b=8 (with a=1 and b=1 as prior)
A normal distribution that is moved towards the right, the highest point being around .65
What is underfitting?
When a model is too simple to capture the data, meaning it will not be a good predictor and replicable structures are taken as noise
What is overfitting?
When the model is too complex for the data, it will not be a good predictor and it will take noise and make it a replicable structure
Occam’s razor?
Answers that require the least amount of assumptions are usually best
What’s the deal with risky predictions?
They gain a large boost in credibility when evidence is found for it, but are also very easily discredited if the case was the other way around
See this as betting on one horse (aka lose or win all) vs. betting a little bit on all horses
The risk reward relationship is established, why is this not the case for complex models?
Complex models have to predict too many things, (again horse betting)
What is the bayes factor? (words)
Ratio of predictive performance
What does bayes factor do opposed to opposed to the p value?
p only looks at the H0, so “one side of the coin”, you either reject or you keep it. Bayes compares two hypothesis and sees under which the data is most likely (H0 vs H1)
What’s BF10 and B01 and what colours in the pizza pie indicate H1 and H0?
BF10 = H1 (red in pizza pie) and BF01 = H0 (white in pizza pie)
What does evidence of absence mean and what does absence of evidence mean?
evidence of absence = the data is providing more evidence to count the H0 more likely
absence of evidence = there is no evidence leaning either way, the data isn’t likely under either hypothesis
What are three advantages of Bayes factor (NOT evidence of absence shit)?
- Quantifies evidence instead of forcing an all or none-decision
- Allows evidence to be monitored as data comes in
- Applies to data from the real world for which no sampling can be articulated (idk wtf this point means help)
There is a large pitfall with bayes factor, called the bayes rate fallacy, what is this?
Interpretation of bayes is wrong. BF01 = 3 means that the data is 3 times more likely under H0 than H1, however people interpret it as H0 being 3 times more likely than H1
What is the bayes factor dependent on, besides yakow, data?
BF is dependent on the question asked (aka changing the likelihood to be more precise or to distribute bets over many likelihoods) and the prior distribution you use
Bayes factor can range a lot, what does BF10 = 1 mean?
Absence of evidence, data is equally likely under either hypothesis
Bayes factor can range a lot, what does BF01 = 13 mean?
There is strong evidence pointing towards the data being more likely under the H0
Bayes factor can range a lot, what does BF10 = 6 mean?
There is moderate evidence pointing towards the data being more likely under the H1
Bayes factor can range a lot, what does BF01 = 47 mean?
There is extremely strong evidence for the data being more likely under the H0
Bayes factor can range a lot, what does BF10 = 2 mean?
Anecdotal evidence for data being more likely under the H1
Bayes factor can range a lot, what does BF01 = 167 mean?
Extreme evidence for data being more likely under H0
