13. Simple Effects Flashcards
What are main effects?
An overall, or average, effect of a condition
E.g.
Is there an effect of Treatment averaged over Hospital?
Is there an effect of Hospital averaged over Treatment?
What are simple contrasts/effects?
An effect of one condition at a specific level of another
Is there an effect of Hospital for those receiving Treatment A? (…and so on for all combinations.)
What are interactions?
A change in the effect of some condition as a function of another
Does the effect of Treatment differ by Hospital?
With effects coding, we can also think of this as a difference in simple effects.
How do you find simple effects in r?
Use emmeans
- Use pairs () in R
What does the marginal mean, mean?
It is the average of mean values
What do beta coefficients represent in effects interactions (with contrasts)?
Differences between marginal mean and grand mean
Aim: Whether the distance of the row or column marginals from grand mean differs depending on level of another condition
With the example of the hospital data set, what would each coefficient mean?
yijk = beta 0 + (b1E1 + b2E2) + b3E3 + (b4E13 +b5E23) + Ei
Treatment Hospital Interactions
b0 = Grand mean
b1 = Difference between row marginal for treatment A & grand mean
b2 = Difference between row marginal for treatment B & grand mean
b3 = Difference between column marginal for hospital 1 and grand mean
b4 = Does the effect of treatment a differ across hospital 1 and hospital 2?
b5 = Does the effect of treatment b differ across hospital 1 and hospital 2?
How can you find the overall effect of the interaction?
Compare empty model that doesn’t include interaction and then compare model with interaction in anova
What do pairwise tests compare?
compare all levels of a given predictor variable with all levels of the other
Why do we use pairwise comparisons?
Sometimes we don’t have a concrete hypothesis to test, even if we do = explanatory analysis (still some useful information for the field)
What our type I error in a single test?
Set at 0.05
How does type I error differ in multiple tests?
P(Type I error) = a
P(X making Type I error) = 1-a
P(Not making Type 1 in m) = (1-a)m
P(making type I error in m) = 1 - (1-a)m
What is the family wise error rate?
P(making type I error in m) = 1 - (1-a)m
Increased family (set of tests) = Increased chance of a false positive
What are the different types of corrections we can use to keep the type I error rate at the set alpha rate?
Bonferroni
Sidak
Tukey
Scheffe
How do Bonferroni and Sidak keep the type I error rate at the set alpha rate?
Conservative adjustments
Treats individual test in family like they’re independent
It’s moving alpha by how many tests we’re conducting
Bonferroni alpha = alpha/number of tests
To use this value, we would compare the exact
p-value of a particular test to the adjusted α
Alternatively, we can adjust the p-value itself, and then compare the adjusted p-value to the original α
This is what emmeans does.
Sidak = 1-(1-a)1/number of tests
How does Scheffe keep the type I error rate at the set alpha rate?
Scheffe = Calculates p-value from F-distribution
- Adjustment related to the number of comparisons made
- Makes F-value increased for a fixed alpha
Essentially makes the critical value of F larger for a fixed α, dependent on the number of tests.
The square-root of the adjusted F provides an adjusted t
How does Tukey keep the type I error rate at the set alpha rate?
Tukey’s HSD
- Compares all pairwise group means.
- Each difference is divided by the SE of the sum of means.
- This produces a q statistic for each comparison.
- And is compared against a studentized range distribution.
How does using anova() for a single model, and for the model comparison approach yield slightly different results?
Sums of squares difference is the same
Degrees of freedom are the same
F is slightly different for Treatment and Hospital (and therefore so is p-value)
Note the main conclusions do not change.
This difference relates to differences in the degrees of freedom associated with the F-test.
What is a benefit of Bonferroni compared to Tukey?
One benefit of Bonferroni is that it can be applied to any set of P-values, whereas Tukey only applies when comparing the means of levels of a factor. The downside, however, is that it may be overly conservative (i.e. reduce our power to detect an effect that is truly there).
What is the issue of multiple comparisons?
The more tests we do, the higher chance of a type 1 error (false-positive)
