Lecture 24: Chance 2 Flashcards
What are p values?
Probability of getting study estimate (or one further from the null), when there is really no association, just because of sampling error (chance)
If probability really low, then unlikely that estimate is due to sampling error (chance)
What is Ho vs Ha?
the null hypothesis (H0)
Really is no association in the population - Parameter equals null value
Ratio Measures
(RR, OR) = 1
Difference Measures
(RD) = 0
the alternative hypothesis (HA)
Really is an association in the population - Parameter does not equal null value
Ratio Measures
(RR, OR) ≠ 1
Difference Measures
(RD) ≠ 0
What would the full interpretation of the following be?
Study of association between oral contraceptive use and ovarian cancer (large study)
OR = 2.1, p = 0.01
Woman who used oral contraceptives were 2.1 times as likely to develop ovarian cancer compared with those who had never used oral contraceptives
The probability of a OR of 2.1 or further from the null, when the null hypothesis is true, is 0.01
Since the p-value is less than 0.05 the association is statistically significant. We reject the null hypothesis and accept the alternative hypothesis. Chance is an unlikely explanation of the study finding
What would the full interpretation of the following be?
Study of association between oral contraceptive use and ovarian cancer (small study)
OR = 2.1, p = 0.15
Woman who used oral contraceptives were 2.1 times as likely to develop ovarian cancer compared with those who had never used oral contraceptives
The probability of a OR of 2.1 or further from the null, when the null hypothesis is true, is 0.15
Since the p-value is greater than 0.05 the association is not
statistically significant. We fail to reject the null hypothesis and reject the alternative hypothesis. The study finding is consistent with chance as an explanation
What is type 1 error?
When the study finds an association (accepting Ha) when the truth is there is no association (actually Ho)
Example:
In a medical study, this error would mean concluding that a new drug works when it actually does not.
What is type 2 error?
When the study finds no association (failing to reject Ho) when the truth is there is an association (should accept Ha)
Example:
In a medical study, this error would mean concluding that a new drug doesn’t work when it actually does.
How does sample size effect the p value?
Bigger sample size = more likely to get small p
Smaller sample size = less likely to get small p
How can statisticians minimise the chance of a type 2 error?
Statisticians can calculate power to find out how many
participants are needed to minimise chance of a Type-II error
(sample size)
Why p-values are problematic?
Arbitrary threshold:
Statistical significance threshold is arbitrary and artificial, Always useful to report p-values rather than just ‘statistically significant’ or ‘not statistically significant’
Only about H0:
- Don’t say anything about precision
- Just give evidence about consistency with the null hypothesis (Best presented with confidence intervals
Nothing about importance:
- Don’t say anything about whether the results are valid,
useful or correct
- Statistical significance is not clinical significance
- Absence of a statistically significant association is not
evidence of absence of a real association
If the 95% confidence interval does not include the null value, suggest a plausible value for the corresponding p-value.
The confidence interval does not include the null value, so the finding is statistically significant at the 5% level. The p-value will be less than 0.05.
Suppose a study on the relationship between high fruit juice consumption (compared to low/no fruit juice consumption) and the development of dental
cavities reported the following findings:
RR = 2.3 (95% CI = 0.9-3.7, p = 0.06).
Interpret these findings
People with high fruit juice consumption are 2.3 times as likely to develop dental cavities compared to people with low/no fruit juice consumption.
We are 95% confident that the true relative risk is between 0.9 and 3.7. As the confidence interval includes the null value of 1, this finding is not statistically significant. The finding is consistent with chance as an explanation.
The probability that this result, or one further from the null, would occur when there truly was no association in the underlying population is 0.06. As this is greater than 0.05, the finding is not statistically significant and is consistent with chance as an explanation. Therefore, we fail to reject the null hypothesis
What might the researchers do to get a smaller p-value?
Include more people in the study
