Bayesian Analysis- not a full question, just a few marks

Question 1

what is bayesian hypothesis testing?

Answer 1

• Short version: A version of statistical inference that allows us to show a level of evidence for either (i.e., alternative or null) hypothesis
  
  • Show evidence for effect or no effect
    Show evidence for a level of effect (weak, strong etc)

Question 2

null hypothesis testing vs bayesian hypothesis significance testing

Answer 2

• Null Hypothesis Significance Testing (NHST) (standard hypothesis testing) is:
  ○ Absolute
  ○ Decision-based
  ○ Objective
  ○ Solitary
  
  • Bayesian Hypothesis Testing (BHT) is:
    ○ Relative
    ○ Strength-based
    ○ Subjective
    Cumulative

Question 3

absolute vs relative evidence

Answer 3

• NHST focuses on absolute evidence
  ○ We assess how unlikely the data are under the null hypothesis
  ○ If the data are too unlikely, we reject the hypothesis outright (must be an effect- our alternate)
  
  • BHT focuses on relative evidence
    ○ We assess how likely the data are under each hypothesis
    ○ We select the hypothesis that is better supported by the data
    ○ Rather than just rejecting one and accepting another instead we say which one is a better account of the data
  • The difference may seem minor, but it has important implications
    ○ Case of Sally Clark is a bad usage of absolute evidence- convicted of murdering of her two syndrome as they died of SID syndrome
    ○ Stats expert said that statistically it is too unlikely that two children could die of sudden infant death that she must be guilty
    Did not consider how likely a mother is to murder her children- this proves her innocence

Question 4

decision based vs strength based

Answer 4

• NHST focuses on making a decision about the null
  ○ We make a decision about whether or not to reject the null hypothesis
  ○ As N -> ∞, smaller and smaller effects will result in the null being rejected
  ○ As you get an infinite amount of data, smaller and smaller results would result in the null being rejected
  
  • BHT focuses on expressing the relative strength of evidence
    ○ We assess how likely the data are under each hypothesis
    ○ We can use the Bayes factor to express the strength of evidence
  • The difference may seem minor, but it has important implications
    Lindley’s paradox- null hypothesis is rejected when Bayesian will show you it does not have to be rejected

Question 5

objective vs subjetive probability

Answer 5

• NHST focuses on situations where probability is seen as objective
  ○ The probably is determined by repeating an identical situation as many times as possible, and is the same for everybody
  
  • BHT allows for subjective beliefs, and probabilities for one-off events
    ○ Each person can have a specific belief of how likely something is, and these beliefs can be compared once we observe the data
  • The difference may seem minor, but it has important implications
    ○ Under a frequentists objective framework it does not matter as probability is objective- does not matter who makes the judgment
    In a framework like Bayes you can get probabilities based on different opinions

Question 6

solitary vs cumulative science

Answer 6

• NHST focuses on the data within the current experiment
  ○ The assessment is largely independent of any information that we have from prior research
  
  • BHT provides a natural way of incorporating previous evidence
    ○ We can express how likely we think different hypotheses are before having seen the data, which can be based on previous research
    ○ Gives you an overall scape for the amount of evidence for different hypotheses
    The difference may seem minor, but it has important implications

Question 7

key aspects of bayesian stats

Answer 7

• Prior model odds and prior distribution often referred to as the priors
  ○ Think about this is a belief or knowledge you have
  
  • Prior model odds are essentially the amount of evidence you have before looking at the data for each hypothesis
  • Prior distribution is at the parameter level
  • Bayesian information is done by having predictions from prior distribution- if you think there is going to be no difference between groups this would be a difference.
  • Look at the actual data
  • This allows us to go in 2 different ways: hypothesis testing (p value, can you reject anything) and then parameter estimation (getting things like effect sizes)
  • Bayes factor- ley hypothesis testing tool which tells you the relative evidence for the alternate and null hypothesis
  • Posterior distribution is of a parameter
    Cumulative aspect- cycle where you can keep reusing the data you have for different experiments

Question 8

key aspects of NHST

Answer 8

• No prior distribution just a null hypothesis which leads to predicting (allowing us to figure out how likely the data is under the hypothesis)
  Leads to data and p values (hypothesis test) and effect sizes (parameter estimation)

Question 9

prior distribution

Answer 9

Prior distribution
- Expressing beliefs about what the parameter will be (e.g. probability of heads, effect size between groups)
- Have one of these for each of our hypotheses
Used to see what we are going ro test against our null

Question 10

prior predictive distribution

Answer 10

• Data expressed based on the prior distribution
  Different priors make different predictive distributions e.g. if we think that the coin is biased towards heads then our distribution will be biased

Question 11

bayes factor

Answer 11

• How likely the data are under the hypotheses
  
  • Similar to a p value
  • Compares the data under the hypotheses
  • Probability under the null and alternate divided by each other
  • Interpret the Bayes factor using the levels of evidence (weak, moderate or strong)
    ○ BF10: Bayes factor of 1 means there’s the same amount of evidence for both the alternate and the null
    ○ Bayes factor greater than 1-infitie means you have evidence for the alternate (coin being biased
    Bayes Factor less than 1- no effect which is evidence for the coin being fair

Question 12

posterior distribution

Answer 12

• Prior distribution x what we learned from the data
  
  • Based on the data you observe the prior will update to get the posterior
    Key thing is the credible interval

Question 13

credible interval

Answer 13

• Similar to a confidence interval
  
  • 95% sure that the true value is in a range
    Based on a posterior distribution

Question 14

in text example write up

Answer 14

• Just the Bayesian analysis:
  ○ ‘A Bayesian independent samples t-test was conducted to compare the effect of Teacher on Statistics Exam Score, demonstrating moderate evidence for no effect of Teacher (BF10 = 0.1907).
  
  • Both together:
    ‘An independent samples t-test was conducted to compare the effect of Teacher on Statistics Exam Score, demonstrating a significant difference (t(9998) = 2.07, p = .039) between Graeme (M = 65.3, SD = 9.96) and Nathan (M = 64.89, SD = 10.04). However, a Bayesian independent samples t-test demonstrated moderate evidence for no effect of Teacher (BF10 = 0.1907).’