Hypothesis testing Flashcards
Hypothesis testing
The act of evaluating empirical evidence in order to determine the level of support for the hypothesis vs the null hypothesis
T-tests
T-tests are generally used to compare the means of two groups. The greater the difference in means, the further the value of t will be from zero. The following are variations of t-tests: student’s t-test, paired student’s t-test, welch’s t-test, and one-sample t-test.
Paired student’s t-test
The paired student’s t- test is used when comparing the means of two related groups (e.g., pre- and post-test scores for the same individuals). The samples are therefore not independent but rather paired or matched in some way. It assumes that the differences between pairs of observations are normally distributed in the dependent variable.
Student’s t-test
Sometimes referred to as ‘independent samples t-test’, and it compares the means of two independent groups. It assumes that the data are normally distributed and have equal variances. (Often just called t-test)
Welch’s t-test
Welch’s test is for comparing two independent groups, but it does not assume equal variances. It is more robust than the Student’s t-test when the variances are unequal.
One-sample t-test
This tests whether the mean of a single sample is significantly different from a known or hypothesized mean.
Non-parametric alternatives to t-tests
When the assumption of normality is violated, non-parametric tests can be used. The following are types of non-parametric tests: Wilcoxon Signed-Rank Test, and Mann-Whitney U Test.
Wilcoxon signed rank test
This is the non-parametric alternative to the paired t-test. It is used to compare two related samples or repeated measurements on a single sample. It assesses whether the population mean ranks differ.
Mann-Whitney U test
This is the non-parametric alternative to the independent samples t-test. It is used to compare two independent groups and assesses whether one group tends to have larger values than the other.
ANOVA and its variants
ANOVA (Analysis of Variance) is used to compare the means of three or more groups. The following are types of ANOVA: One-way ANOVA, Welch’s F-test, Friedman Test, and Kruskal-Wallis Test.
One-way ANOVA
This is used when there is one independent variable with three or more levels and one dependent variable. It assumes that the data are normally distributed and have equal variances across groups.
Welch’s F-test
Similar to Welch’s t-test, Welch’s F-test (or Welch’s ANOVA) does not assume equal variances and is a more robust alternative to the one-way ANOVA when this assumption is violated.
Friedman test
This is the non-parametric alternative to the repeated measures ANOVA (used when the same subjects are measured multiple times). It is used to test for differences between groups when the dependent variable is ordinal or not normally distributed.
Kruskal Wallis test
This is the non-parametric alternative to the one-way ANOVA. It is used to compare three or more independent groups and assesses whether the population distributions are identical. Used when the dependent variable is skewed/does not have a normal distribution.
Post-hoc tests
If an ANOVA or Kruskal-Wallis test shows a significant difference between groups, post-hoc tests are used to determine which specific groups differ from each other. Common post-hoc tests include:
* Equal variance assumed
o Duncan (best for equal group sizes)
o Hochberg’s GT2 (best for unequal group sizes)
* Unequal variance assumed
o Games-Howell (regardless of group sizes)
