Significance tests and p values Flashcards
Definition of a null hypothesis
Staring assumption about the population of interest
No real difference/association
Definition of a p value
The probability that you will get AT LEAST the observed value assuming that the null is true
A way of weighing the evidence against null
Probability from a significance test
The larger the difference observed in your sample, the less likely that it could have come from a population where no difference exists
Definition of clinical significance
Difference observed is large enough to be clinically meaningful
Not a statistical judgement
Why do we use significance tests
Results are based on a sample, can’t be sure if results benefit whole population
Different samples give different results, could be due to chance
What is the null hypothesis
Assumes that there is no difference/association
Must have enough evidence from sample data to reject null
What is the p value
Why is it useful
Probability found from statistical significance tests
Helps you weigh up evidence against null
the larger the difference observed in your sample, the less likely that is could have come from a population where no difference exists
How do you interpret p < 0.05
If p< 0.05 => reject null as association exists => statistically significant
If 0 not included in CI, can really reject null
How do you interpret p ≥ 0.05
If p ≥ 0.05 => can’t reject null => not statistically significant
Can’t accept it, other possibilities ar present
If 5% within 95% for difference => results are consistent with null
How do you present p values
State exact p values
Avoid only stating p< 0.05 or p≥ 0.05
-only acceptable if p is tiny
Allows for full interpretation of evidence
What is the difference between clinical and statistical significance
When would a result be clinically important but not statistically?
When would a result be statistically important but not clinically?
Difference observed is large enough to be clinically meaningful
Clinically important, not statistically => occurs when sample size is small
Statistically important, not clinically => occurs when sample size is large
What is a 2 sided significance test
When would you use it
Used when you have a
- null hypothesis, no difference
- alternative hypothesis, difference exists on either side
What is a 1 sided significance test
When would you use it
Used when you have a
- null hypothesis, no difference/difference exists in 1 direction
- alternative hypothesis, difference exists in other direction
How would you choose between a 2 sided and 1 sided test
1 sided does not differentiate between no effect and a harmful effect => unaccepatble
2 sided should always be use unless clearly justified before hand
