MANOVA & ANCOVA Flashcards
When should you use a MANOVA
You can think of MANOVA as essentially performing several between subjects ANOVAs as a single set.
You would do this when you have multiple related outcomes (DVs) you want to assess at once
You have one or more between-subjects predictors (IVs)
And you want to control the Type I error rate for the set of DVs
The MANOVA is necessary only when you have a theoretical reason to consider all your DVs as a set
Without such a reason, just do separate ANOVAs
Name the four assumptions of MANOVA
- The residuals follow a multivariate normal probability distribution, with means equal to zero (check the Q-Q plot).
- The variance-covariance matrices of each group of residuals are equal (check Box’s M test).
- The observations are independent (by row, like with MR).
- The DVs are correlated, but not too correlated (below ±.90).
Which statistics should we look for in a MANOVA analysis
The Wilks’ Lambda is the one most often reported for MANOVA
Conveniently, if you calculate 1-Lambda it becomes a proportion of variance explained statistic, like the R2 for regression
Pillai’s Trace is considered safer in most cases, so if Wilks’ Lambda and Pillai’s Trace disagree you should worry about the accuracy of Wilks’ Lambda
They typically do agree, however, and if they do you might as well accept convention and use Lambda
What should you do if there is a significant Wilk’s Lambda
The typical follow-up in this case is a Univariate ANOVA; one for each Dependent Variable
This is also done for us by jamovi, though not fully
A significant univariate ANOVA needs to again be followed-up with simple effects tests (planned comparisons) as usual
What should we keep in mind to control Type 1 error rate with post hoc MANOVA analyses
We want to control our Type I error rate when we perform multiple tests, as with our multiple Univariate ANOVAs and their follow-up mean comparisons.
jamovi will not adjust its Univariate ANOVA F s for us, nor is there isn’t a way to do all the post-hoc comparisons within MANOVA (we likely would not want to anyway)
To control Type I error, apply a simple Bonferroni correction to alpha
Divide your desired alpha (.05) by the number of DVs in the MANOVA
This becomes your new cut-off for all Univariate ANOVA p-values
E.g. alpha = 0.5/3 = .017
What is a problem you may encounter with MANOVA post hoc significance
Your MANOVA can sometimes be significant despite none of the following univariate ANOVAs showing significance.
This is because those ANOVAs don’t control for the DV dependencies
Performing a Discriminant Analysis is the solution to this problem, and it’s probably what should actually be done for any MANOVA, but jamovi doesn’t provide this option (nor did SPSS for that matter)
Essentially, it identifies first whether related groups of variables differ and then how they differ from each other
As usual, it is possible to do using R code in jamovi
For our purposes, just knowing it exists is good enough!
When should we use ANCOVA
- We have a categorical variable that slices up our DV into comparison groups (an ANOVA predictor)
- We also have a non-categorical variable we think will help explain some extra variance in the DV
What is the goal of running an ANCOVA
When we add the covariate in an ANCOVA we are trying to reduce just the error variance in the DV.
This means we are increasing the probability that the IV can predict a significant amount of the remaining variance, because there’s less variance left to predict
It does not matter whether the covariate is a significant predictor or not in this case
Why should we run an ANCOVA (and why not)
We should choose to run an ANCOVA when we have a potential covariate measured, and:
DO:
We really want to have the best estimate of our IV’s effectiveness
DON’T
We already ran an ANOVA, and it wasn’t significant
The covariate is meant to explain only the variance otherwise unexplained by the ANOVA model.
So, if your intended covariate is not independent of your categorical predictor, then you should not use it
What assumption should you check for in ANCOVA
For an ANCOVA to be valid you also need to have homogeneity in your covariate/DV regression slopes, for each of your categorical predictor groups.
The lines of best fit need to be pretty much the same
There are two ways to verify you do not have a problem here:
1. Plot a Covariate x DV regression line for each level of your predictor (the visual test method)
- Temporarily add an IV/Covariate interaction term to your ANCOVA model (the statistical test method). If it’s significant, it fails.
How should you get the means of ANCOVA variables
It is important to remember that you must interpret the estimated marginal means from the ANCOVA rather than doing a simple descriptive analysis to get the means.
The estimated marginal means have regressed the predictor on the outcome, while including the covariate
If you calculated the means any other way than the covariate’s effect would be dropped, and the means would be different
