Comparisons & Contrasts Flashcards
Type 1 error
claim ONE effect when isn’t ONE
Type 2 error
TO retain Null when there is an effect
Name the 2 types of type 1 error…
2 types of type 1 error:
1) Per comparison error rate (PC) – probability of making a type 1 error on any comparison
2) Familywise error rate (FW) – probability of making type 1 error in a family of comparisons
What is FW error rate if alpha=0.05 and 5 comparisons are made?
1 in 4 chance you get a type 1 error.
1 - (1 - .05)^5 =
what are the two possible ways of doing more specific analyses after an ombibus?
post hoc and a priori
what are the two myths about multiple comaprisons ?
1) Overall omnibus has to be significant–! Not true, not the end.
2) Using planned comparisons means you don’t need corrections against type 1 error inflation rate
if the overall F is sig then …
Ð Requiring overall significance will change the alpha level for FW errors, making the MC conservative (when overall F must be sig, then some subsequent tests become more conservative)
Ð MCs often tackles the actual hypothesis more directly
Methods for a priori comparisons?
1) Multiple t tests
2) Linear contrasts
3) Weighting coefficients for comparisons
two types of comparisons ?
orthogonal
non-orthogonal
multiple t test issues?
Simple but problematic in large number of comparisons and planned in advance
Needs homogeneity of variance, when there is a violation or unequal sample sizes, use Welch test
how does one Weighting coefficients for comparisons
?
Assign a weight to each mean. 0 for means left out of comparisons. At least need non-0.
Means to be contrasted are assigned opposite weights group 1 = -1 group 2 = +1
Weights must sum to 0.
E.G. > treatment vs placebo would be High (1/3) Med (1/3) Low (1/3) Placebo (-1) = 0
Always comparing two groups – can be comprised of many as long as sums to 0.
E.G. Comparing groups 1-3 and 4-5:
1 (1/3) 2 (1/3) 3 (1/3) 4 (-1/2) 5 (-1/2)
what is an orthogonal comparison?
Orthogonal = indepenedant of one another – can’t use same group twice. Can’t compare IV1 against IV2 and then IV1 against Iv3
Number of possible contrasts is K-1(df) e.g. 3 = 2 orthogonal contrasts //// 8 = 7 contrasts.
orthogonal comparison rules?
RULE: Has to be non-overlapping variance … i.e. has comparisons have to analyse non-overlapping variance. A set of contrasts that are mutually independent of one another. If a group is singled out in one comparison, it should not reappear in another comparison. Each contrast must compare only 2 chunks of variance.
First comparison = EXP vs control groups
Second Comparison = Within EXP or control groups
E.G contrast 1 =(Low and High Dose vs Placebo) contrast 2 = (low vs high dose)
If one is significant, it has no bearing on the rest.
If you want to do more, you need corrections to be made
what is an non-orthogonal comparison?
Contrasts NOT independent of each other
E.G
Contrast 1: group 1 vs group 2 (excluding group 3)
Contrast 2: group 1 vs group 3 (excluding group 2)
However, p-values may be correlated
corrections for non-orthogonal ?
BONFERRONI t
Boole’s inequality: The probability of occurrence of at least one of a set of events can never exceed the sum of their individual probabilities. Bonferroni set bounds on this inequality
Bonferroni corrects the alpha level based on the number of comparisons & evaluates t against Dunn’s table. So based on number of comparisons, the alpha level is reduced, say from .05 to .01. Alternative test is Dunn-Sidak test, which is a variation on Bonferroni but super similar – no clear benefit as so similar. Dunn- Sidak has greater stat power in large numbers.
Multistage Bonferroni = Holm procedure – for controlling FW error rates for multiple hypothesis.
