Topic 8: Multiple Comparisons Flashcards
What is an error rate?
The probability of making a Type 1 error for a specific set of analyses
What is alpha-pc?
Error rate per comparison: probability of making a Type 1 error for each individual comparison
Set by the researcher
What is alpha-ew?
Error-rate per experiment / experiment-wise error: probability of a Type 1 error for a family of comparisons
Involves a uniquely computed table of critical values
What is a post hoc comparison?
Tests between sample means not specified before data were collected
Assumes all possible pairwise comparisons will be made
Used for exploratory analysis
What is a planned comparison?
Subset of comparisons planned and justified by researcher before data are collected
What are the features of planned comparisons?
Significant f(obs) - not required
Must justify choice of comparisons before data are collected
Total # of comparisons cannot exceed k-1
How do you set alpha-pc?
Divide alpha by the number of comparisons you are making - Bonferroni correction
Why is MSerror used instead of s-squared p for t(obs)-Dunn’s?
It is a better estimate of population error
What is the question posed by a Dunne’s test?
What is the minimum distance between 2 means for t(crit)-Dunn’s?
What are the features of post hoc comparisons?
Significant f(obs) is required
Adjust a for all possible comparisons (k*[k-1]/2)
Less powerful - harder to reject null
What are the number of comparisons possible post hoc?
k*(k-1)/2
When should Tukey’s HSD test be used, and what does it require?
Best for k>3
Homogeneity of pop variance cannot be violated
Requires significant f(obs)
What statistics does Tukey’s HSD test use?
Q statistics - Qhsd
What does Qhsd mean?
The minimum difference value
How is Tukey HSD written in APA?
Post-hoc probes using Tukey HSD revealed that the comparisons between X and Y, X and Z, and Y and Z were (not) significantly different, with the critical value for Qhsd = +/-W
