Corrections (generally for non-orthogonal)

Question 1

Boole’s inequality

Answer 1

The probability of one of a set of events can never exceed the sum of their individual probabilities

Bonferonni set bounds on this:

e.g. 3 comparisons, each with probability of Type 1 error a=.05
so overall the probability of a Type 1 error CANNOT exceed (.05+.05+.05)= .15

Question 2

Bonferonni correction

Answer 2

a/c (c= number of comparisons)

e.g. a=.05, c=5 so .05/5=.01 use this as the significance threshold (a=.01)

Question 3

Problems with bonferonni correction

Answer 3

:( the more comparisons done, the lower the alpha is so it is harder to observe significant effects (reduced Type 1 error but increases Type 2)

:( it can be overly conservative

Question 4

Holm procedure (multistage bonferroni)

Answer 4

For controlling the FW error rate

1) Scale t for all contrasts, arrange the t values in rank order
2) Check the largest first against the critical value in Dunn’s (1961) table corresponding to contrasts (=a/c)
3) if it is significant, the next largest t has an alpha based on c-1

E.g. 5 groups:

1st: a=.05, c=5, so .05/5=.01 (like bonferonni)
2nd: a=.05, c=4, so .05/4=.0125
3rd: a=.05, c-3, so .05/3=.0167
etc. stop when a non significant result is researched (minimised number of analyses, might not do all 5)

Question 5

Benefit of the Holm procedure

Answer 5

:) more LIBERAL threshold for each successive comparison, you are more likely to find a significant effect

Question 6

when to use FDR

Answer 6

When there are lots of significant effects, it is better to control proportion of errors (false discovery rate), rather than the probability of making one error (familywise error rate)

Question 7

FDR (false discovery rate)

Answer 7

Used to control for FW error rate (Benjamini & Hochberg, 1995)

V= number of true null hypotheses rejected (false positives)
R= total number of nulls rejected
FDR= V/R if R>0

If R=0 (there are no significant effects) so don;t bother with it

Question 8

Where does the new criterion come from in FDR

Answer 8

The new criterion is based on number of tests significant, relative to their rank.

1) rank p values in order
2) look at the table with ranked p values, find where p

Question 9

advantages and disadvantages of FDR

Answer 9

:) more power than FW error corrections
:) you can bear more errors when there are more nulls rejected

:( higher rates of a Type 1 error than FW error corrections