Lecture 4: Hypothesis testing Flashcards
Hypothesis testing
A method of inferential statistics which allows researchers to draw conclusions about the population based on sample data. It involves formulating hypotheses, calculating test statistics, determining p-values and drawing conclusions about the null hypothesis
Equality hypotheses vs inequality hypotheses
Equality hypotheses state that a value, difference or effect is equal to zero.
Inequality hypotheses state that a value, difference or effect is larger or smaller than a specific value.
Hypothesis testing does not provide evidence for hypotheses, but rather helps casting doubt on a null hypothesis.
Fisher’s philosophy on hypotheses
Suggests only using a null hypothesis. If this null hypothesis is rejected, the “truth” must be anything other than the null hypothesis.
Neyman-Pearson’s philosophy on hypotheses
Suggests stating specific null and alternative hypotheses, with an explicit expected effect size for the alternative hypothesis. This allows for calculating the probabilities of drawing correct or incorrect conclusions.
Type I error
Refers to rejecting the null hypothesis when it is true: a false-positive conclusion. This risk is also called Alpha
Type II error
Refers to accepting the null hypothesis when it is false: failing to detect a true effect. This risk is also called Beta.
In general, Beta (B) decreases when the effect size is greater, the sample is larger and there is less “noise” (lower standard deviation).
Steps involved in hypothesis testing
- Formulate hypotheses: this involves stating a testable proposition about population parameters
- Calculate a test statistic: a test statistic describes how many standard errors away from the population statistic, under the null hypothesis, the sample statistic is
- Calculate the p-value: the p-value represents the probability value of observing a value at least as extreme as the sample statistic, assuming the null hypothesis is true
- Draw a conclusion about the null hypothesis: based on the p-value, we either reject or fail to reject the null hypothesis.
What do p-values show?
The probability of observing certain data assuming the null hypothesis is true. The p-value is then compared to a pre-determined significance level (denoted as alpha) to make a decision about accepting or rejecting the null hypothesis.
Rejecting the null hypothesis indicates that the observed data is unlikely to occur if the null hypothesis were true. Failing to reject the null hypothesis means that the observed data is not surprising or does not provide sufficient evidence to reject it.
T-distribution
With the Z-distribution, we use the population’s standard deviation. The problem is that we rarely know the population’s standard deviation. Using our sample’s SD introduces uncertainty as SD is not exactly equal to the standard deviation. If we don’t account for this uncertainty, p-values will be too small. Therefore, we use the T-distribution that gives slightly larger p-values.
Degrees of freedom
Controls how thick the tails of our distribution are.
Lower df: thicker tails, higher p-values for the same test-statistic
Higher df: skinnier tails, lower p-values for the same test statistic
As the degrees of freedom increase, the more the distribution tends towards the standard normal distribution. Usually this happens when the sample is bigger or equal to 30.
Degrees of freedom can be calculated by using the formula n - 1
Critical values
The Z-values corresponding to the chosen α level. We reject the null hypothesis when the test statistic exceeds the critical value.
- For a two-sided test and α = 0.05, the critical value is 1.96
- For a one-sided test and α = 0.05, the critical value is 1.64
Power
Probability of correctly finding a true effect (you’ll literally hold power then because you’re great)
