Week 3 - ANOVA II

Question 1

• Explain why significance tests are not that helpful when we want to determine the importance of findings 
(x3)

Answer 1

Binary decision tells us little about importance of result - loss of info
Or the practical significance
Can get anything to be significant with a big enough sample size

Question 2

• Define effect sizes and explain why they are useful (x3) 


Answer 2

Estimate the magnitude of experimental effect
Another way of assessing the reliability of the result in terms of variance
Allow comparison of effects within designs (or across if standardised)

Question 3

o Identify what eta-2 estimate describes
 (x2, plus describe calculation x2)

Answer 3

Proportion of variance in the sample’s DV scores that is accounted for by the effect
1= 100% of variance
SSeffect
Divided by SStotal

Question 4

o Identify what omega-2 estimate describes
 (x1, plus describe calculation x3)

Answer 4

Proportion of variance in the population’s DV scores that is accounted for by the effect
Variability of effect
Minus error variance (weighted by the degrees of freedom),
Divided by the total variability plus error

Question 5

o Explain which of eta2/omega2 is more biased than the other (and why)
 (x1, x1)

Answer 5

Eta-2 is more biased/larger than omega-2 (more conservative/likely to be replicated)
Because omega-2 it accounts for error in estimating the population

Question 6

o Identify the factors that influence the difference between the eta-2 and omega2 (and how)

Answer 6

In omega-2, Ms error is weighted by defect, and removed from SSeffect
While MSerror is added to SStotal in the denominator

Question 7

• Explain the difference between eta-squared and partial eta-squared in terms of what each estimate describes 


Answer 7

Eta-2 is proportion of total variance accounted for by effect - SStotal
Partial is proportion of residual variance accounted for by the effect - SSeffect
o Residual variance = variance left over to be explained
• (i.e., not accounted for by any other IV in the model)

Question 8

• Identify the two key limitations of partial eta-squared (x2, x1)

Answer 8

In factorial ANOVA, [error + effect] is less than [total], so partial-η2 is more liberal/inflated - larger than eta-2
o Unless other factors in model don’t account for any variance, eg other factor and interaction variance = 0

In factorial ANOVA, η2 adds up to a maximum of 100%, but partial eta2 can add to > 100%

Question 9

• Explain what an omnibus test is, and list the effects in a 2-way ANOVA that can be 
categorized as omnibus tests

Answer 9

Omnibus tests look for all possible differences
o So a main effect means there’s at least one difference among marginal means
o And interaction is at least one difference in the simple effects

Question 10

• Identify the circumstances in which follow-up tests are required in 2-way factorial ANOVA 
(x2)

Answer 10

When an omnibus test is significant
(and then on, if the follow-ups are also significant)
And factor has more than 2 levels

Question 11

• When following up main effects in a 2-way ANOVA: 


o Identify the research question being tested
 (x1)

Answer 11

Which marginal means differ from the other/s?

Question 12

• When following up main effects in a 2-way ANOVA: 


o Identify the type of means that are compared
 (x1)

Answer 12

Marginal means

Question 13

• When following up main effects in a 2-way ANOVA: 


o Identify the type of statistical test (e.g. F, t, z) that is used
 (x3)

Answer 13

Protected t-test - pairwise comparisons protected against Type I inflation
Linear contrasts - single or set of means differ from each other
Simple pairwise tests (SPSS auto)

Question 14

• When following up main effects in a 2-way ANOVA: 


o Explain how you would determine the number of follow-up tests that are needed
 (x2)

Answer 14

Does the factor have >2 levels? If not, report

If so, need main effect comparisons on marginal means involved in significant main effects

Question 15

• When following up main effects in a 2-way ANOVA: 


o Explain what a significant main effect comparison tells you (x1)

Answer 15

Where the differences in marginal means lies within that factor

Question 16

• When following up a significant two-way interaction in a 2-way ANOVA:
o Identify the two (possible) steps in the analyses


Answer 16

Simple effects

Simple comparisons

Question 17

• When following up a significant two-way interaction in a 2-way ANOVA:
o Identify the types of means that are being compared. 


Answer 17

Cell means

Question 18

• When testing simple effects in a 2-way ANOVA: 


o Identify the research question being tested
 (x2)

Answer 18

Where do the differences in cell means lie?

*What are the effects of one factor (focal IV) at each level of the other factor?

Question 19

• When testing simple effects in a 2-way ANOVA: 


o Identify the type of statistical test (e.g. F, t, z) that is used
 (x1, plus describe x2)

Answer 19

F-test

• SStreat (For factor A at level j of Factor B)
• Divided by MSerror from omnibus test

Question 20

• When testing simple effects in a 2-way ANOVA: 


o Identify the error term that is used


Answer 20

MSerror from original omnibus test

Question 21

• When testing simple effects in a 2-way ANOVA: 


o Identify how degrees of freedom are calculated for SSeffect / treatment
and SSerror (x2, x1)

Answer 21

df for effect/treatment is (#levels of the focal variable - 1) (from original source table)
dferror is dferror from original source table error

Question 22

• When testing simple effects in a 2-way ANOVA: 

o Explain how you would determine the maximum number of simple effects that could be tested for a factor
 (x2, plus e.g. x3)

Answer 22

From hypothesis/theory,
And then, as many as there are levels of the second factory
In a 3x2 ANOVA:
Simple effects of A = tests at level 1 and 2 of B
Simple effects of B = tests at levels 1/2/3 of A

Question 23

• When testing simple effects in a 2-way ANOVA: 


o Explain what a significant simple effect tells you (x1)

Answer 23

That there are differences among the cell means of a factor, at each level of another

Question 24

• In a 2-way ANOVA, explain how the variance is re-partitioned when testing:
o The simple effects of Factor A
 (x3)

Answer 24

Simple effects re-partition the main effect and interaction variance
Variance from main effect of Factor A would be add to that of the interaction
*Because this = totals of SSa at b1, b2, b3 etc
Then repartitioned for A at B1, A at B2…

Question 25

• In a 2-way ANOVA, explain how the variance is re-partitioned when testing:
o The simple effects of Factor B (x3)

Answer 25

Simple effects re-partition the main effect and interaction variance
Variance from main effect of Factor B would be add to that of the interaction
Because this = totals of SSb at a1, a2, a3 etc
Then repartitioned into B at A1, B at A2 etc…

Question 26

• Identify the circumstances in which simple effects would need to be followed up (x2)

Answer 26

When they are significant

And have >2 levels

Question 27

• When testing simple comparisons…


o Identify the research question (and hypothesis) being tested (x1)

Answer 27

Where do the differences in simple effects lie?

Question 28

• When testing simple comparisons…


o Identify the type of statistical test (e.g. F, t, z) that is used (x3, x1))

Answer 28

t-tests
Linear contrasts
Pairwise comparisons
(same as main effect comparisons, but with cell, not marginal means)

Question 29

• When testing simple comparisons…


o Explain what a significant simple comparison tells you

Answer 29

Which cell means/effects of combined IVs are different from the others

Question 30

• Identify the problems (and solutions) associated with simple comparisons


Answer 30

SPSS gives you all possible comparisons - redundant info
*Resolve with orthogonal linear contrasts
Increased family-wise error
*Bonferrroni adjustment for critical t
*Do a priori tests (ie fewer)

Question 31

• Be sure to know and understand the steps for following up main effects and 2-way interactions that are presented in the flow-charts.

Answer 31

a

Question 32

What are Cohen’s conventions for differentiating/describing effect sizes? (x3)
But? (x1)

Answer 32

0. 2 = small
1. 5 = medium
2. 8 = large

Normally in psych ours are a fraction of this - 1-5%

Question 33

Which effect size should you report in factorial ANOVA? (x3, x1, x1, x1)

Answer 33

Partial-eta2
• Convenience - automatic in SPSS
• Bias – larger effect size, = more likely to be published

Eta square is better, and usually makes most sense to use it
Unless small sample, or large variance, in which omega
• Omega

Question 34

What are the considerations for when to use a Bonferroni adjustment to critical t in follow-up tests?

Answer 34

o Whether you decided to do the comparisons “a priori” or “post hoc”
o How many comparisons you’re conducting – 5+
o Whether you want to be “conservative” or “liberal”