Hypothesis Testing Flashcards
What is the null hypothesis?
There is no statistical significant difference between specified population, any observed difference is due to sampling or experimental error (chance).
What do we ask when we conduct a hypothesis test?
1) is the value of the test statistic extreme enough for us to reject the null hypothesis?
2) what is our test statistic
3) what should the distribution of the test statistic be if the null hypothesis is true.
What is the p-value?
The probability of obtaining a particular value of the test statistic or a more extreme value if the null hypothesis is true.
What is alpha?
This is a pre determined level of significance which enables us to decide whether to reject the null hypothesis.
If the p-value is <= alpha we regard it as significant band reject the null hypothesis.
If the p-value is >= alpha we fail to reject the null hypothesis
What is a Type I error?
This is when the null hypothesis is actually true but we reject it.
What is a Type II error?
This is when the null hypothesis is false but we fail to reject it.
What is the probability of making a Type I error?
This is equal to the value of alpha (significance threshold)
Write a null hypothesis and a hypothesis for a one sample T-test?
Null hyp: the mean of this sample is equal to the hypothesised value
Hyp: The mean of this sample is different from the hypothesised value.
In a T-test in which the null hypothesis is true where should the value of t come from?
The value of t should come from a t- distribution with n-1 degrees of freedom
When would you use a Paired T-test?
When there is a pair of measurements per subject and you are measuring a before and after change.
(We still only have n independent data points even if we have 2n numbers)
What distribution is tested in a paired T-test
Difference between pairs of values are calculated and then distribution of differences is tested to see if it is equal to 0.
(It’s a one sample T-test where the hypothesised value is always zero)
What is a general null hypothesis and hypothesis for a 2 sample T-test.
Null hyp: the means of these two samples are the same
Hyp: the means of these two samples are different
(We are looking at the difference between two mean)
(Difference between two means also has a sampling distribution)
What does a 2 sample T-test assume?
1) The data is normally distributed
2) The variances of the two groups are equal
If the null hypothesis is true where should the t value come from?
1) For a single sample T-test
2) For a 2 sample T-test
The t-value should come from a t- distribution with n-2 degrees of freedom (where n is the total number of data points from both groups)
1) When is a 2 sample T-test robust?
2) What are the exceptions?
1) When sample sizes of both groups are >= 30 (provided groups show equal variances)
2) if each group being compared have similar numbers in each group then it’s ok to use the test even when the standard deviations in the two groups differ by up to 3 fold
What is a non parametric substitute for the 2 sample T-test?
Mann Whitney Wilcoxon test
State general null hypothesis and hypothesis for Analysis of Variance (ANOVA)
Null hyp: All of the means of these groups are the same
Hyp: at least one of these groups has a different mean from the others
When is an Analysis of Variance (ANOVA) used?
Used to test the null hypothesis on a data set with 2 or more groups.
What is the test statistic of an ANOVA?
F ratio
What is the test statistic for T-tests?
T value
What is the F ratio?
F= the treatment Mean square/ Error Mean square
Therefore a ratio between 2 variances
What is the Treatment Mean square?
This is the explained variation (signal)
What is the Error mean square?
This is the unexplained variation (noise)
What is the total sum of squares (in ANOVA)?
This is the treatment sum of squares + the error sum of squares
What is the total degrees of freedom (in ANOVA).
Treatment degrees of freedom + error degrees of freedom
What determines which f distribution to use for the null distribution?
Total degrees of freedom
How are mean squares constructed?
These are constructed by dividing the sum of squares by the degrees of freedom.
How do you calculate the treatment mean of square?
Treatment sum of squares/Treatment degrees of freedom
How do you calculate the Error mean square?
Error sum of squares/Error degrees of freedom
What are F ratios defined by?
1) Whichever number of degrees of freedom are associated with top of F ratio fraction.
2) whichever number are associated with the bottom of the F ratio
1) When aren’t F ratios significant?
F ratios aren’t significant when they are <= 1
An f ratio of 1 means the amount of noise is equal to the amount of variation
What are Chi squared tests used for?
Chi squared is used to check interactions between count data.
You test whether the distribution of counts for different levels of one factor are consistent with/ have an effect across levels of another factor
1) What are the assumptions of a Chi squared test?
2) what is generally accepted?
1) The expected counts must be >= 5
2) If counts aren’t >= 5 it is generally accepted as long as all the expected counts are >= 1 and 80% of them are >= 5.
Why is >= 5 important in Chi squared?
Chi squared is constructed to make the assumption that the data follows a normal distribution and that can only be done for a Poisson distribution (count data) if the mean is at least 5.
What is a common correction that is used when testing multiple hypotheses on the same data?
Bonferroni correction
What is the Bonferroni correction?
If you have an overall significance threshold of alpha and you carry out m hypothesis tests. To reject the null hypothesis for any of the tests the p value must be <= a/m
What are the assumptions of ANOVA?
1) Assumes residuals are normally distributed.
2) Assumes variance big unexplained variation is constant throughout the data set ( homogeneity of residuals).
How would you correct for right skew?
1) Root data
Or
2) log data
How can you correct for left skew?
1) squaring data
What are parametric tests?
Tests that rely on pre defined distributions e.g normal distribution
What are non parametric tests?
Tests that do not rely on specific pre defined distributions
However they still make some assumptions
What is a general null hypothesis for the Mann Whitney Wilcoxon Test?
Two groups being compared come from the same distribution, with the same median but it does not matter what the shape of the distribution is?
Therefore the 2 groups being compared must have similar skews.
