Week 3 - ANOVA II Flashcards
• Explain why significance tests are not that helpful when we want to determine the importance of findings (x3)
Binary decision tells us little about importance of result - loss of info
Or the practical significance
Can get anything to be significant with a big enough sample size
• Define effect sizes and explain why they are useful (x3)
Estimate the magnitude of experimental effect
Another way of assessing the reliability of the result in terms of variance
Allow comparison of effects within designs (or across if standardised)
o Identify what eta-2 estimate describes (x2, plus describe calculation x2)
Proportion of variance in the sample’s DV scores that is accounted for by the effect
1= 100% of variance
SSeffect
Divided by SStotal
o Identify what omega-2 estimate describes (x1, plus describe calculation x3)
Proportion of variance in the population’s DV scores that is accounted for by the effect
Variability of effect
Minus error variance (weighted by the degrees of freedom),
Divided by the total variability plus error
o Explain which of eta2/omega2 is more biased than the other (and why) (x1, x1)
Eta-2 is more biased/larger than omega-2 (more conservative/likely to be replicated)
Because omega-2 it accounts for error in estimating the population
o Identify the factors that influence the difference between the eta-2 and omega2 (and how)
In omega-2, Ms error is weighted by defect, and removed from SSeffect
While MSerror is added to SStotal in the denominator
• Explain the difference between eta-squared and partial eta-squared in terms of what each estimate describes
Eta-2 is proportion of total variance accounted for by effect - SStotal
Partial is proportion of residual variance accounted for by the effect - SSeffect
o Residual variance = variance left over to be explained
• (i.e., not accounted for by any other IV in the model)
• Identify the two key limitations of partial eta-squared (x2, x1)
In factorial ANOVA, [error + effect] is less than [total], so partial-η2 is more liberal/inflated - larger than eta-2
o Unless other factors in model don’t account for any variance, eg other factor and interaction variance = 0
In factorial ANOVA, η2 adds up to a maximum of 100%, but partial eta2 can add to > 100%
• Explain what an omnibus test is, and list the effects in a 2-way ANOVA that can be categorized as omnibus tests
Omnibus tests look for all possible differences
o So a main effect means there’s at least one difference among marginal means
o And interaction is at least one difference in the simple effects
• Identify the circumstances in which follow-up tests are required in 2-way factorial ANOVA (x2)
When an omnibus test is significant
(and then on, if the follow-ups are also significant)
And factor has more than 2 levels
• When following up main effects in a 2-way ANOVA:
o Identify the research question being tested (x1)
Which marginal means differ from the other/s?
• When following up main effects in a 2-way ANOVA:
o Identify the type of means that are compared (x1)
Marginal means
• When following up main effects in a 2-way ANOVA:
o Identify the type of statistical test (e.g. F, t, z) that is used (x3)
Protected t-test - pairwise comparisons protected against Type I inflation
Linear contrasts - single or set of means differ from each other
Simple pairwise tests (SPSS auto)
• When following up main effects in a 2-way ANOVA:
o Explain how you would determine the number of follow-up tests that are needed (x2)
Does the factor have >2 levels? If not, report
If so, need main effect comparisons on marginal means involved in significant main effects
• When following up main effects in a 2-way ANOVA:
o Explain what a significant main effect comparison tells you (x1)
Where the differences in marginal means lies within that factor
• When following up a significant two-way interaction in a 2-way ANOVA:
o Identify the two (possible) steps in the analyses
Simple effects
Simple comparisons
• When following up a significant two-way interaction in a 2-way ANOVA:
o Identify the types of means that are being compared.
Cell means
• When testing simple effects in a 2-way ANOVA:
o Identify the research question being tested (x2)
Where do the differences in cell means lie?
*What are the effects of one factor (focal IV) at each level of the other factor?
• When testing simple effects in a 2-way ANOVA:
o Identify the type of statistical test (e.g. F, t, z) that is used (x1, plus describe x2)
F-test
- SStreat (For factor A at level j of Factor B)
- Divided by MSerror from omnibus test
• When testing simple effects in a 2-way ANOVA:
o Identify the error term that is used
MSerror from original omnibus test
• When testing simple effects in a 2-way ANOVA:
o Identify how degrees of freedom are calculated for SSeffect / treatment and SSerror (x2, x1)
df for effect/treatment is (#levels of the focal variable - 1) (from original source table)
dferror is dferror from original source table error
• When testing simple effects in a 2-way ANOVA:
o Explain how you would determine the maximum number of simple effects that could be tested for a factor
(x2, plus e.g. x3)
From hypothesis/theory,
And then, as many as there are levels of the second factory
In a 3x2 ANOVA:
Simple effects of A = tests at level 1 and 2 of B
Simple effects of B = tests at levels 1/2/3 of A
• When testing simple effects in a 2-way ANOVA:
o Explain what a significant simple effect tells you (x1)
That there are differences among the cell means of a factor, at each level of another
• In a 2-way ANOVA, explain how the variance is re-partitioned when testing:
o The simple effects of Factor A
(x3)
Simple effects re-partition the main effect and interaction variance
Variance from main effect of Factor A would be add to that of the interaction
*Because this = totals of SSa at b1, b2, b3 etc
Then repartitioned for A at B1, A at B2…
• In a 2-way ANOVA, explain how the variance is re-partitioned when testing:
o The simple effects of Factor B (x3)
Simple effects re-partition the main effect and interaction variance
Variance from main effect of Factor B would be add to that of the interaction
Because this = totals of SSb at a1, a2, a3 etc
Then repartitioned into B at A1, B at A2 etc…
• Identify the circumstances in which simple effects would need to be followed up (x2)
When they are significant
And have >2 levels
• When testing simple comparisons…
o Identify the research question (and hypothesis) being tested (x1)
Where do the differences in simple effects lie?
• When testing simple comparisons…
o Identify the type of statistical test (e.g. F, t, z) that is used (x3, x1))
t-tests
Linear contrasts
Pairwise comparisons
(same as main effect comparisons, but with cell, not marginal means)
• When testing simple comparisons…
o Explain what a significant simple comparison tells you
Which cell means/effects of combined IVs are different from the others
• Identify the problems (and solutions) associated with simple comparisons
SPSS gives you all possible comparisons - redundant info
*Resolve with orthogonal linear contrasts
Increased family-wise error
*Bonferrroni adjustment for critical t
*Do a priori tests (ie fewer)
• Be sure to know and understand the steps for following up main effects and 2-way interactions that are presented in the flow-charts.
a
What are Cohen’s conventions for differentiating/describing effect sizes? (x3)
But? (x1)
- 2 = small
- 5 = medium
- 8 = large
Normally in psych ours are a fraction of this - 1-5%
Which effect size should you report in factorial ANOVA? (x3, x1, x1, x1)
Partial-eta2
• Convenience - automatic in SPSS
• Bias – larger effect size, = more likely to be published
Eta square is better, and usually makes most sense to use it
Unless small sample, or large variance, in which omega
• Omega
What are the considerations for when to use a Bonferroni adjustment to critical t in follow-up tests?
o Whether you decided to do the comparisons “a priori” or “post hoc”
o How many comparisons you’re conducting – 5+
o Whether you want to be “conservative” or “liberal”
