Significance Tests Flashcards
What is the null hypothesis?
States the two treatments are equally effective.
Purpose of a significance test?
Uses the sample data to assess how likely to null hypothesis is to be correct.
E.g. ‘there is no difference in the prevalence of colorectal cancer in patients taking low-dose aspirin compared to those who are not’.
What is the alternative hypothesis?
The opposite of the null hypothesis –> i.e. there is a difference between the treatments.
What is the p value?
The probability of obtaining a result by chance at least as extreme as the one that was actually observed, assuming that the null hypothesis is true.
It is therefore equal to the chance of making a type I error (see below).
What are the two types of errors that may occur when testing the null hypothesis?
1) Type I
2) Type II
What is a type I error?
The null hypothesis is rejected when it is true.
I.e. Showing a difference between two groups when it doesn’t exist, a false positive.
This is determined against a preset significance level (termed alpha).
Is the chance of making a type I error affected by sample size?
No - as the significance level is determined in advance
What increases the chance of making a type I error?
If the number of end-points are increased.
For example if a study has 20 end-points it is likely one of these will be reached, just by chance.
What is a type II error?
The null hypothesis is accepted when it is false.
i.e. Failing to spot a difference when one really exists, a false negative.
What is the probability of making a type II error termed?
Beta
What is the probability of making a type II error determined by?
Sample size & alpha
What error is made when:
The study accepts the null hypothesis (H0) but the reality is the alternative hypothesis (H1)?
Type II error (beta)
As the study as failed to spot a difference when one really exists (false negative).
What error is made when:
The study rejects the null hypothesis (H0) and the reality is the null hypothesis (H0)?
Type I error (alpha)
As the study has shown a difference between two groups when one doesn’t exist (false positive).
What is the power of a study?
The probability of (correctly) rejecting the null hypothesis when it is false
i.e. the probability of detecting a statistically significant difference
Formula for power?
Power = 1 - the probability of making a type II error
What can power be increased by?
Increasing the sample size
What does the type of significance test used depend on?
Where the data is parametric (something which can be measured, usually normally distributed) or non-parametric.
What are the 2 types of parametric significance tests?
1) Student’s t-test: paired or unpaired
2) Pearson’s product-moment coefficient: correlation
What does paired data refer to?
Data obtained from a single group of patients e.g. measurement before and after an intervention.
What does unpaired data refer to?
Comes from two different groups of patients e.g. comparing response to different interventions in two groups
What are the 4 types of non-parametric significance tests?
1) Mann-Whitney U test
2) Wilcoxon signed-rank test
3) Chi-squared test
4) Spearman, Kendall rank
What is the chi-squared test used for?
Used to compare proportions or percentages e.g. compares the % of patients who improved following two DIFFERENT interventions.
Purpose of significance tests?
To determine if the results of an experiment support or reject a null hypothesis (H0), and if an alternative hypothesis (H1) can be accepted
What is a Wilcoxon signed rank test used for?
Compares two sets of observations on a single sample, e.g. a ‘before’ and ‘after’ test on the same population following an intervention.
What is a Spearman’s rank correlation coefficient and Kendall’s rank correlation coefficient used for?
measure the strength and direction of the association between two variables
