Hypothesis testing Flashcards
What is a type I error?
The probability that we will reject the null hypothesis (concluding that there is an effect) when there actually isn’t
A false positive
Thinking you have found gold but it is actually an aluminium can
What is the typical level of type I errors in biological sciences?
0.05 / 5%
What is a type II error?
Failing to reject a false null hypothesis (when there is an effect)
A false negative
A metal detector doesn’t beep when there is gold underground
Test statistic for one-sample t-test
t = difference between measured mean and hypothesised value of the mean / SE of the mean
What does a one-sample t-test do?
Tests whether the mean of the sample of data is equal to a hypothesised mean
Paired t-test
Special case of the one-sample t-test
We have a pair of measurements per subject (before and after for example)
What does a paired t-test do?
Tests whether the mean of the set of differences is equal to zero
Two-sample t-test test statistic
t = difference between out measured difference between 2 means and the difference between 2 means that we expect under the null hypothesis / SE of the difference between 2 means
What does the two-sample t-test do?
Tests if the means of two samples of data are the same or significantly different
Two-sample t-tests assuming equal variances are robust for sample sizes…
> = 30
Per group
If the two groups have _____ in each group, then it is still OK to use the two-sample t-test when the SDs of the groups differ by up to ______ fold
Similar numbers
Three
What is the t-test where equal variances are not assumed?
Welch’s T-test
What is the test statistic for an ANOVA?
F = treatment mean square / error mean square
(explained variance (signal) / unexplained variance (noise))
What does an ANOVA do?
Tests if all the means of more than two groups are the same or significantly differ
What do hypothesis tests involve?
A test statistic
A distribution from which we expect the test statistic to come from if the null hypothesis is true
Calculation of a p-value
What does a p-value tell us?
The probability of obtaining the value of the test statistic, or a more extreme value, if the null hypothesis is true
What is an interaction?
When the effect of one variable on the outcome variable differs depending on the level of another variable
The effect of a drug on blood pressure differs depending on the type of diet they eat
Assumptions of a t-test
Data are normally distributed
Assumptions of ANOVA
The error (residuals) is normally distributed
Variance of the unexplained variation is constant throughout the dataset (homogeneity of error/residuals)
Chi Squared Test statistic
χ2 = sum( (observed - expected)2 / expected )
How do you calculate the expected value in Chi Squared?
Expected = (row total x column total) / grand total
Assumptions of a Chi Squared Test
All expected counts are >= 5
Mean of counts = the variance of the counts
The Poisson distribution can be approximated by a Normal distribution, which is true as long as the mean is >= 5
If all counts aren’t >= 5 in Chi Squared Test, when is it still reasonable to run the test?
As long as all expected counts are >= 1
80% of the expected counts are >= 5
How to check for normally distributed residuals?
Plot histogram of the residuals and check the shape
Plot the normal Q-Q plot of the residuals (the dots should be not too far from the line, shouldn’t bend away from the line in any systematic fashion)
