Stats - hypothesis testing Flashcards
Why is there often error in hypotheses testing?
Generally speaking, it is not possible to investigate hypotheses on entire populations. As such, one usually takes samples and uses these to make estimates about the population from which they are drawn. This leads to uncertainty as there is no guarantee that the sample taken will be truly representative of the population. Such samples therefore involve potential for error.
What is the null hypothesis? (H0)
This states that there is no difference or relationship between the variables or groups in the study
What is the alternative hypothesis? (HA or H1)
What is the meaning of 1-tailed and 2-tailed?
The alternative hypothesis (Indicated by H1 or Ha) is the opposite of the null hypothesis. This is sometimes referred to as the research hypothesis. This suggests that any difference is due to some non-random chance i.e. the difference is real.
The alternative hypothesis can be one or two-tailed.
If the alternative hypothesis simply seeks to establish a difference then it is two-tailed. For example you may wish to investigate if the mean weight of people has changed over the past 10 years. In this case you are not suggesting that people have increased or decreased in weight but simply changed.
If you are seeking to investigate a change in something in one direction then this is referred to as one-tailed. For example, you may suspect that clozapine therapy increases peoples’ weights. You are not just investigating if clozapine changes peoples’ weight but are also suggesting that it changes it in one particular direction (that is up in this case).
What is Type 1 error when testing a null hypothesis?
What would a p-value of 0.04 represent in this case?
Type I: the null hypothesis is rejected when it is true (finding a difference that didn’t exist), this is also called a false positive. This is determined against a pre-set significance level (termed alpha). As the significance level is determined in advance the chance of making a type I error is not affected by sample size.
i.e. Rejecting a true null hypothesis (false positive)
P value of 0.04 = 4% probability that the observed difference is due to chance.
What is Type 2 error when testing a null hypothesis?
What would a p-value of 0.04 represent here?
Type II: the null hypothesis is accepted when it is false (failing to find a difference that really existed), this is also called a false negative. The probability of making a type II error is termed beta. It is determined by both sample size and alpha
i.e. FAILING TO REJECT a false null hypothesis (false negative)
P value of 0.04 = 4% chance of concluding there is no difference when one exists
What is the Power of a study?
The power of a study is the probability of (correctly) rejecting the null hypothesis when it is false
Power = 1 - the probability of a type II error
Power can be increased by increasing the sample size
What is the significance level? (alpha)
This is a pre-defined “cut-off” level. This relates to how sure you need to be (it’s just like in the courts where you have a ‘beyond reasonable doubt’ and ‘balances of probabilities’). The cut-off is called the significance level (aka alpha level). Typically, this cut-off is set at 0.05 (this is the 5% level where there is a less than 1 in 20 chance of being wrong).
What is the p-value (probability value)?
P-values provide information on statistical significance. They help us decide if study results have occurred due to chance. It does NOT take into account bias.
The p-value is the probability of obtaining a result that is as large or larger when in reality there is no difference between two groups.
Alternatively you can understand this as the probability of rejecting the null when it is true.
What do the different p-values represent?
- high
- low
- greater than or equal to significance level
- less than significant level
The p-value can take on any number from 0 to 1 (as it is a probability). A high p-value would indicate that there is a high chance that an observed difference is due to chance and a low p-value that there is a low probability that the observation is due to chance.
If the p-value is found to be less than the cut-off then you reject the null hypothesis.
If the p-value is greater or equal to the cut-off then you fail to reject the null.
NB: p-values may show that study is statistically significant but not necessarily clinically significant
What is the Bonferroni Correction?
What does it reduce?
What does it increase?
The Bonferroni correction is a statistical method used to address the issue of multiple comparisons in hypothesis testing.
When performing multiple tests on the same dataset, the likelihood of committing a Type I error (false positive) increases with the number of comparisons made. The Bonferroni correction helps control this by adjusting the significance level (alpha) for each individual test.
The correction works by dividing the overall significance level (typically 0.05) by the number of tests being conducted. For example, if five tests are performed, the new significance level for each test would be 0.05/5 = 0.01.
By using the Bonferroni correction, the overall probability of committing a Type I error across all tests remains at the desired alpha level, reducing the chance of false positives.
HOWEVER it does reduce the chance of false negatives, as it becomes harder to identify true effects at smaller significance levels
i.e less TYPE 1 error, more Type 2 error
