Planned Comparisons & Post-hoc Tests

Question 1

Why do we use follow-up tests?

Answer 1

• ANOVA tells us there is a significant effect but not which conditions are significantly different from each other
• Two options:
  
  • Planned comparisons: when we have a specific hypothesis
  • Post-hoc tests: don’t need to have a specific hypothesis
• Only use when our main ANOVA analysis has found a significant difference

Question 2

Planned comparisons

Answer 2

Break down variance found in ANOVA and test specific hypotheses about which groups’ scores vary

Question 3

Planned comparisons rules

Answer 3

1. Each contrast should only compare two chunks of variance
2. Each contrast should test a unique hypothesis - contrasts should not overlap with each other

Question 4

Coding guide for assigning weights in SPSS

Answer 4

1. Groups given positive weights are always compared to groups given negative weights (“only compare two chunks”)
2. Sum of weights for each contrast = 0
3. Any group compared on its own should not be used in any further contrasts (i.e. contrasts should not overlap)
4. Groups assigned a weight of zero are not included in that comparison

Question 5

Planned comparisons SPSS output

Answer 5

Question 6

Post-hoc tests

Answer 6

• Don’t need a specific hypothesis beforehand
• Compare each mean against each other
• Control for family-wise error
• More conservative than planned comparisons → higher chance of type II error
• Options
  
  • Conservative: less likely to make a type I error but more likely to make a type II error
    
    • Bonferroni
    • Scheffe
  • Less conservative: less likely to make a type II eror but more likely to make a type I error
    
    • Tukey HSD
  • When homogeneity of variance assumption is violated
    
    • Games-Howell

Question 7

Bonferroni by hand

Answer 7

• Stricter criterion to reach significance

Question 8

Bonferroni SPSS output

Answer 8