13. Simple Effects

Question 1

What are main effects?

Answer 1

An overall, or average, effect of a condition

E.g.
Is there an effect of Treatment averaged over Hospital?
Is there an effect of Hospital averaged over Treatment?

Question 2

What are simple contrasts/effects?

Answer 2

An effect of one condition at a specific level of another

Is there an effect of Hospital for those receiving Treatment A? (…and so on for all combinations.)

Question 3

What are interactions?

Answer 3

A change in the effect of some condition as a function of another

Does the effect of Treatment differ by Hospital?
With effects coding, we can also think of this as a difference in simple effects.

Question 4

How do you find simple effects in r?

Answer 4

Use emmeans

• Use pairs () in R

Question 5

What does the marginal mean, mean?

Answer 5

It is the average of mean values

Question 6

What do beta coefficients represent in effects interactions (with contrasts)?

Answer 6

Differences between marginal mean and grand mean

Aim: Whether the distance of the row or column marginals from grand mean differs depending on level of another condition

Question 7

With the example of the hospital data set, what would each coefficient mean?

yijk = beta 0 + (b1E1 + b2E2) + b3E3 + (b4E13 +b5E23) + Ei
Treatment Hospital Interactions

Answer 7

b0 = Grand mean
b1 = Difference between row marginal for treatment A & grand mean
b2 = Difference between row marginal for treatment B & grand mean
b3 = Difference between column marginal for hospital 1 and grand mean
b4 = Does the effect of treatment a differ across hospital 1 and hospital 2?
b5 = Does the effect of treatment b differ across hospital 1 and hospital 2?

Question 8

How can you find the overall effect of the interaction?

Answer 8

Compare empty model that doesn’t include interaction and then compare model with interaction in anova

Question 9

What do pairwise tests compare?

Answer 9

compare all levels of a given predictor variable with all levels of the other

Question 10

Why do we use pairwise comparisons?

Answer 10

Sometimes we don’t have a concrete hypothesis to test, even if we do = explanatory analysis (still some useful information for the field)

Question 11

What is the issue of multiple comparisons?

Answer 11

The more tests we do, the higher chance of a type 1 error (false-positive)

Question 11

What our type I error in a single test?

Answer 11

Set at 0.05

Question 12

How does type I error differ in multiple tests?

Answer 12

P(Type I error) = a
P(X making Type I error) = 1-a
P(Not making Type 1 in m) = (1-a)m
P(making type I error in m) = 1 - (1-a)m

Question 13

What is the family wise error rate?

Answer 13

P(making type I error in m) = 1 - (1-a)m

Increased family (set of tests) = Increased chance of a false positive

Question 14

What are the different types of corrections we can use to keep the type I error rate at the set alpha rate?

Answer 14

Bonferroni
Sidak
Tukey
Scheffe

Question 15

How do Bonferroni and Sidak keep the type I error rate at the set alpha rate?

Answer 15

Conservative adjustments

Treats individual test in family like they’re independent

It’s moving alpha by how many tests we’re conducting

Bonferroni alpha = alpha/number of tests

To use this value, we would compare the exact
p-value of a particular test to the adjusted α
Alternatively, we can adjust the p-value itself, and then compare the adjusted p-value to the original α
This is what emmeans does.

Sidak = 1-(1-a)1/number of tests

Question 16

How does Scheffe keep the type I error rate at the set alpha rate?

Answer 16

Scheffe = Calculates p-value from F-distribution

• Adjustment related to the number of comparisons made
• Makes F-value increased for a fixed alpha

Essentially makes the critical value of F larger for a fixed α, dependent on the number of tests.

The square-root of the adjusted F provides an adjusted t

Question 17

How does Tukey keep the type I error rate at the set alpha rate?

Answer 17

Tukey’s HSD
- Compares all pairwise group means.
- Each difference is divided by the SE of the sum of means.
- This produces a q statistic for each comparison.
- And is compared against a studentized range distribution.

Question 18

How does using anova() for a single model, and for the model comparison approach yield slightly different results?

Answer 18

Sums of squares difference is the same
Degrees of freedom are the same
F is slightly different for Treatment and Hospital (and therefore so is p-value)

Note the main conclusions do not change.

This difference relates to differences in the degrees of freedom associated with the F-test.

Question 19

What is a benefit of Bonferroni compared to Tukey?

Answer 19

One benefit of Bonferroni is that it can be applied to any set of P-values, whereas Tukey only applies when comparing the means of levels of a factor. The downside, however, is that it may be overly conservative (i.e. reduce our power to detect an effect that is truly there).