post hoc tests, simple main effects, and planned comparisons Flashcards
Sheffé’s test
a type of post hoc test that can be used if groups have different sample sizes
less sensitive to departures from the assumption of normality and equal variances in the population
most conservative test (very unlikely to reject H0) - good if you want to avoid type 1 errors, but has lower power
how does sheffé’s test work?
- uses F ratio to test for a significant difference between any two means
- but with a larger critical value
- crit val determined by finding as usual and then multiplying it by k-1
- SS are calculated based on which groups are being compared and the residual SS is simply the SS residual from the main analysis
to carry out the test,
we calculate the SS, MS, and F comparison (for the stuff we want to look at)
then compare to the larger crit val - if bigger, then the pair of means is sig different
how to conduct sheffé’s test for one way between subjects ANOVA
- calculate SS, MS, and observed F for each pairwise mean comparison
- groups being compared receive coefficients of -1 and 1; the group(s) excluded from this comparison receive a coefficient of 0
Tukey’s HSD test
- used if groups have equal sample sizes and all comparisons represent simple difference between two means
- uses studentized range statistic, Q
- Q is compared against a critical value of Q for ⍺=.05, which is associated with k and N-K (Q crit)
sidak post hoc test
adjusts type 1 error rate by the number of tests, but is less conservative
dunnet post hoc test
used when one group (usually the control group) is compared to the other k-1 groups; requires critical values from a specialized table
holm post hoc test
sequential mean comparisons with a readjustment to the nominal type 1 error using the bonferroni correction
fisher hayter post hoc test
following a significant omnibus F ratio, compute a minimum absolute difference between means required for statistical significance based on Q crit with k-1 degrees of freedom and compare the two most discrepant means first; less conservative than Tuker’s HSD
Newman-Keuls post hoc test
starts with the largest mean difference, if the H0 that these means are equal is rejected, the test proceeds until H0 is retained for a pair of means; the minimum absolute difference between means required for statistical significance is recalculated at each step (the number of groups that remains to be compared is reduced by 1 at each step)
duncan
same procedure as newman keuls but using different F crit (same at Sidak’s post hoc test)
what is a simple main effect
a post hoc analysis carried out if the effect of one factor at one level depends on the effect of the other factor
if the significant interaction dominants the main effect, we can ignore the main effects altogether and examine the individual simple main effects separately
what do we do if the interaction is not significant but at least one of the main effects is in a two way interaction?
compare the marginal means
what do we do if the interaction is significant but it is dominated by the main effect in a two way interaction
still understand the main effect first and then evaluate how it changes at the levels of the other factor
what are keppel and wicken’s heuristics for selecting the factor for computation of simple main effects
- choose the factor with the greatest number of levels
- choose a quantitative factor
- choose the factor with the greatest SS for the main effect
- choose an experimentally manipulated factor
how do we control type 1 error when conducting follow up analyses for significant interactions?
keppel and wickens recommend setting ⍺fw to 0.10 for the set of simple main effects because these effects incorporate variation due to both the main effect of the factor for which we are computing simple main effects and the interaction
