Lecture 24 chance 2 Flashcards
Internal validity
Extent to which we can rely on we think that the study findings valid based on how the study was done
3 components - chance, bias, confounding
Chance
Sampling error
P values
Based on hypothesis testing
Probability of getting study estimate (or one further from the null), when there is really no association in population, because of sampling error (chance)
P values telling us about associations about
Differences between groups
Whether or not that difference is sufficiently large, inconsistent with there being no difference of underlying population
Truth
Dont know it
Could be association between exposure and outcome Or no association between exposure and outcome
2 ways to be correct
Find an association when there truly is one Or not find an association when there isnt one
Consistent with the underlying population
Could be wrong in 2 ways
find an association in study (measure of association that does not equal null value) when there truly is no association (population parameter is of null value) Or no association when there is one
P values
Probability from study
Study has got it wrong this way it has found an association when there truly is not one in the population
P values require
Null hypothesis
Alternative hypothesis
Really is no association in the population
Population parameter equals null value (ratio measure null value = 1, Difference measure null value = 0)
Really is an association in the population hypothesis is
Parameter does not equal null value
P values Probability when
null hypothesis is true that our study has found an association when there truly isnt
Type 1 error
Finding an association when there truly isnt one
Set at 5%
Probability less than 5%
sufficiently unlikely that chance is the reason for finding this association
Threshold: 0.05 (5%, or 1 in 20)
Find association by chance
If p < 0.05
reject null hypothesis (no association)
Accept alternative hypothesis (is association in population)
Association is statistically significant
Never accept
null hypothesis
p > 0.05
Fail to reject H0
Reject HA
Association is: ‘Not Statistically Significant’
3 elements
Interpret RR measures of associations
Type 2 error
Study Don’t find association that truly exists
Truth there is an association in population between the exposure and outcome
Due to not having enough people in study
To avoid type 2 error
Sufficient statistical power
Enough people in study
Can calculate how many people needed in study to have a certain level of statistical power
To assess statistical significance
See if 95% CI includes null value
95% CI includes null value
95% CI the study findings are consistent with there being no association in study population
Not statistically significant
p > 0.05
applies to RR RD
95% CI doesnt include null value
Not consistent with no association in population
Statistically significant
p < 0.05
applies to RR and RD
3 steps
Interpret measure of association (RR)
Interpret CI
Statistical association
Arbitrary threshold 0.05
Assess significance
No logic behind it
Still allows for error (type 1 error)
P values Refers to chance
Don’t say anything about whether the results are valid, useful or correct
Study Could be bias, study could be done poorly could still get a statistical significant finding
Don’t tell how good the study is Just tell what the likely impact of chance is
Absence of a statistically significant association is not evidence of absence of a real association
Absence of evidence not the same as evidence of absence
Not finding a pvalue that is statistically significant association is not evidence that there is no association. Its just evidence that you haven’t found it
chance consists of
validity
CI
sampling error
p values
What are p-values?
Probability of getting study estimate (or a study estimate further from the null), when there is really no association, because of sampling error (chance)
If p value probability really low,
estimate is due to sampling error (chance)
Uses logic of hypothesis testing
null hypothesis (H0)
Really is no association in the population
Parameter equals null value
RR, OR = 1
RD = 0
the alternative hypothesis (HA)
Really is an association in the population
Parameter does not equal null value
RR OR = 1
RD = 0
P value less than 0.05
Chance is an unlikely explanation of the study finding
Reject H0
Accept HA
Association is:
‘Statistically Significant’
p-value is greater than 0.05
study finding is consistent with chance as an explanation
Fail to reject H0
Reject HA
Association is:
‘Not Statistically Significant’
Probability of getting study estimate (or an estimate further from the null) when there is really no association because of
sampling error (chance)
Type-II errors
Incorrectly fail to reject H0 when should have
(p should have been < 0.05 but got > 0.05)
due to having too few people in the study
Bigger sample size
more likely to get small p
Smaller sample size
less likely to get small p
You can see whether a p-value is greater or less than 0.05 with a
95% confidence interval
p > 0.05
95% CI includes null value
study finding is consistent with chance as an explanation
Not statistically significant
Fail to reject H0 and reject HA
p < 0.05
not 95% CI includes null value
Chance is an unlikely explanation of the study finding
Statistically significant
Reject H0 and accept HA
Why p-values are problematic
Arbitrary threshold
Only about H0
Nothing about importance
Arbitrary / artificial threshold
p value
At 5% threshold will still find a statistically significant association when there really isn’t one at least one time in twenty (Type-I error)
Only about H0
p value
Just give evidence about consistency with the null hypothesis
Don’t say anything about precision
Nothing about importance
p value
Statistical significance is not clinical significance
Don’t say anything about whether the results are valid, useful or correct
Absence of a statistically significant association is not evidence of absence of a real association
