Lecture 6

Question 1

What is the problem of NHST

Answer 1

NHST is based on the principle of falsification “Scientific theories can only be tested by attempting to falsify them”
“For example, the hypothesis that ‘all swans are white’ can be falsified by observing a black swan.”
Falsification focuses on categorically disproving theoretical predictions rather than confirming them. In NHST, “not being able to falsify a theory (e.g., null hypothesis)” does not mean that the theory is correct – it is just a lack of evidence to falsify!

Question 2

What are the 6 key points of p-values?

Answer 2

Key points:
* P-values can indicate how incompatible the data are with a specified statistical model.
* P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
* Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold
* Proper inference requires full reporting and transparency.
* A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.
* By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis

Question 3

What is the bayesian view point?

Answer 3

Bayesian view point:
A priori knowledge: A proposition is knowable independently of experience. Prior probability : Your belief about a proposition without any data provided
A posteriori knowledge: A proposition is knowable based on empirical evidence or experience. Posterior probability : Your belief about a proposition when data is provided as ‘evidence’.
Posterior is affected by (i) your data and (ii) your prior
Bayesian methods are less sensitive to the issue of small samples if you have a valid and strong prior. If your sample size is large enough to dominate priors, then misspecified prior has less impact.

Question 4

What does the BF explain?

Answer 4

Since the parameter is a fixed constant, there is only one of two possibilities. 100% that the null hypothesis is true or 100% that the null hypothesis is false.
Then what does it mean by the ‘probability’ that the null hypothesis is true in Bayesian statistics? This probability reflects your belief, not the truth.

Question 5

How to interpret BF

Answer 5

When BF is 3 for M2 over M1, your data is three times more likely under M2 than under M1. Equivalently, your data is three times less likely under M1 than under M2
In contrast to p values, BF can provide quantitative evidence for the null hypothesis.