week 9.Post hoc tests

Question 1

Post hoc tests

Answer 1

These are performed after the data has been seen. If Anova tells us there is a significant difference between group means, Post hoc can tell us specifically which group/s differ significantly.

With post hoc comparisons, there are no specific predictions about which group mean will differ significantly from others.

Question 2

a priori

Answer 2

knowledge is independent of experience. ie an experimenter decides to compare control data with 2 different treatment groups, before the data has been collected.

Question 3

a posteriori

Answer 3

knowledge depends on experience or empirical evidence

Question 4

error rate per comparison (PC)

Answer 4

the rejection level set for a particular analysis (eg. usually 0.05)

Question 5

Familywise error rate (FW)

Answer 5

The probability of making at least one type 1 error, across a number (or family) of comparisons.This takes into account the increasing probability of making at least one type 1 error, as the number of comparisons increases.

FW=1-(1-a)c

where a=alpha, the per comparison error rate (usually 0.05)

and c=the number of comparison tests.

Thus if we have a=0.05, and we run 4 comparison tests, FW=0.185 which ≈ (0.05 x 4)

Question 6

Studentised range statistic (q)

Answer 6

q_r=X^-_L-X^-_s/[square root of(MS_error/n)]

where r refers to the number of means in the set (ie 3 if had 3 groups)

X^-_L=largest mean

and X^-_s=smallest mean

n=number of participants per group.

Then consult q-statistic table;

if q_{(r, dferror)} > q_critical, then we reject the null hypothesis and conclude that there is a significant difference between the smallest and the largest mean.

Then can continue to recalculate q, to see if there is a significant differnce between other groups. Note though that if r was initially 3 and there were three groups, the middle one was included in the three, but if then comparing the middle and the largest, there are only 2 groups, and r changes (to 2), as does the degrees of freedom and where find critical q on the table.

Question 7

minimum significant difference (MSD)

Answer 7

Once it has been established that there is a significant difference between the largest and smallest group mean, we want to know if there are other significant differences also eg between the middle and largest group mean?

Tukey’s Honestly Significant Difference Test (HSD)

is more conservative that Studentised range, as uses the q critical from the Studentised q statistic calculuated for the largest and smallest group mean, and then holds the formula constant, so does not change with each new pairing unlike the studentised.. So with Tukey.’s calculate the minimum significant difference, and then if any group means differ by >/=MSD, it is statistically significant.

MSD=q_critical [square root of (MS_error/n)]

Question 8

Tukey’s Honestly Significant Difference test (HSD)