ANOVA Comparisons

Question 1

A priori comparison

Answer 1

Decide before conducting ANOVA to compare some, but not all, group means. Depends on which specific sets of means you theoretically think will be different from one another. Usually theory-driven, decreases likelihood of making a Type I error.
Simple: Compare 2 means (will only do these by hand!)
Complex: Compare 3+ means (will use SPSS)

Question 2

Post-hoc comparison

Answer 2

Compare all means after conducting ANOVA. This is what most researchers do, easier from a calculation perspective.

Question 3

Tukey’s honestly significant difference (HSD) test formula

Answer 3

q(x,y) = M(x) - M(y)/(√MS(within)/n)

Question 4

Tukey test steps

Answer 4

1. Calculate denominator term: (√MS(within)/n)
2. Calculate q(x,y): Code groups (ex. CBT = 1, BA = 2, PDT = 3), then calculate for every possible combination of groups (1,2; 1,3; 2,3). Take the absolute value since we are interested in the difference in magnitude for Tukey tests.
3. Locate q[crit]: use k, df(within), and alpha level to find q[crit] value on the q table. Round DOWN your df(within) if the exact value is not on the table, even if the # is closer to a higher value.
4. Identify significant differences between means: Compare means of the groups to your q[crit] value. If q(x,y) > q[crit], those means ARE significantly different from each other. If q(x,y) < q[crit], those means are NOT significantly different from each other.

Question 5

A priori comparisons steps

Answer 5

1. Calculate MS(between)
2. Calculate F-ratio
3. Complete Bonferroni correction
4. Set the criteria for a decision
5. Make decision

Question 6

Compute MS(between)

Answer 6

1. df(between) = k - 1
2. SS(between)
   a. Calculate Ms: M = Σ(x)/N
   b. Calculate GM: GM = Σ(M)/n, where n = # of means for each group
   c. Calcuate SSs: SS = Σ(M-GM)^2 (ignore the raw scores here!)
   d. Calculate SS(between) = SS1 + SS2
3. Calculate MS(between) = SS(between)/df(between). SS(between) will always = MS(between) for a simple a priori since k = 2, so df = 1.

Question 7

Bonferroni correction

Answer 7

Controls the Type I error rate when conducting multiple comparisons. The Bonferroni corrected alpha becomes the new testwise alpha level. α(new) = α/m, where m = # of tests conducted. Round down to nearest α level on F-table.

Question 8

Testwise alpha level

Answer 8

The risk of type I error for one hypothesis test. Probability = α. Ex: testwise alpha for each z-test, t-test, ANOVA = .05

Question 9

Experimentwise alpha level

Answer 9

Risk of type I error for total # of hypothesis tests conducted. Probability = 1 - (1 - α)^m, where m = # of tests conducted.