me myself and I lecture 3- p-values

Question 1

what is the p-value?

Answer 1

The p-value is the probability that we would have observed a given deviation if the null hypothesis was correct.
A smaller p-value means that there is stronger evidence in favour of the alternative hypothesis.
A p-value of 0.05 or lower is generally considered statistically significant.

Question 2

what does p=0.05 actually mean?

Answer 2

It is essentially a 5% probability, a 1/20 chance that we would have seen a given deviation by random chance if the null hypothesis was true.

So if we think about this in terms of accepting scientific facts, 5% of statements might not be correct

Question 3

what does it mean if the p-value is very small?

Answer 3

we are more inclined to believe that our data diverge from the null hypothesis (do not support the null hypothesis)

we are more inclined to believe that an alternative hypothesis explains our data
Either the null hypothesis is not relevant to our data or there is some bias in how we have collected the data.

Question 4

what is sampling error?

Answer 4

Our interpretation of results is only as good as our experimental design.

A sample is usually only a (very small) subset of the total population.

This can impact our results and must be considered. Data collection and experimental design are very important.

The p-value is a value we can use to determine what is the likelihood that the deviation we see from the null hypothesis is due to sampling error.

Question 5

what is type 1 error?

Answer 5

false positives, p = 0.049
reject H0?

Question 6

what is type 2 error?

Answer 6

false negatives, p = 0.051
accept H0?

Question 7

what are the p-value threshold?

Answer 7

Different institutes, scientific fields, experiments … will have different p-value thresholds.
The p-value is <0.05 but this is open to type I and type II error problems.
* p < 0.003 “evidence”
* p < 0.0000003 “a discovery”
* p = 0.000025 “strong evidence”

Question 8

what is the effect size?

Answer 8

Effect size helps researchers assess the magnitude of an effect e.g. the difference in means between two groups or the strength of an association between variables.

Small effect sizes may indicate minimal or negligible effects, while large effect sizes suggest substantial effects.