ANCOVA

Question 1

How does ANCOVA work?

Answer 1

• ANCOVA just takes any of the existing ANOVAs that we learnt about, and adds a continuous covariate (CV)
  ○ For example, we could have Simple BS ANCOVA, or Complex WS ANCOVA, etc.
  ○ There’s also “MANCOVA”, but let’s never speak of this…
  
  • Traditional ANCOVA just adds a “main effect” of the continuous CV to the model
    ○ We want to account for the effect that our CV is having on our DV
    ○ If we don’t account for the CV, then there will be more unexplained variance in the model, which will make our test less powerful
    ○ We’ll also use a “non-traditional” version of ANCOVA, which includes the interaction between the IV(s) and the CV; this is robust against the violation of one of the assumptions (homogeneity of regression slopes), but is technically just a linear model (i.e., it’s not commonly referred to as ANCOVA)

Question 2

Example

Answer 2

• We want to know whether intoxication influences working memory, but also know from being experts that working memory is known to decrease with age
  
  • A single between-subjects IV of Alcohol Dose with 3 levels:
    ○ Placebo
    ○ Low Dose
    ○ High Dose
  • A single DV of working memory performance on a well-validated working memory task
  • A single CV of age in years, to account for the variability in working memory performance associated with age and soak up some of that error term in our model
  • Based on what you’ve learnt so far, you’d probably just run a one-way ANOVA on the DV, and ignore the CV
  • Instead, the ANCOVA assesses whether your IV has an effect on your DV while “controlling for” your CV
  • In our example: Does alcohol dose effect working memory, while controlling for age?
  • I write “controlling for”, as you need to be careful with how far you take this idea
  • A lot of bad practices in ANCOVA are based on people thinking that they can just “control for” a major confound in their experiment!!
  • Apart from this, you treat it like you would the standard ANOVA without the CV
  • Test main effects/interactions, and follow up with post-hoc tests when needed

Question 3

what is a covariate?

Answer 3

• In short, a (continuous) variable that we think might influence our DV
  ○ Not so different from an IV, though often we’re not really interested in the CV
  ○ However, you need to be careful with your covariate selection!!
  
  • Too many researchers have the “Kobeing” mentality to ANCOVA
    ○ Covariates can’t “fix” design or sampling flaws
    ○ Covariates shouldn’t just be any random variable you can think of
    ○ ANCOVA isn’t a magic wand to make problems go away
  • If we make poor choices of covariates, then we can come to misleading conclusions
    Covariates should be based on knowledge (e.g., literature, expertise, etc.), and be things that we think might influence the DV, while being distinct from our IV(s)

Question 4

covariates should be independent (distinct) from the IVs

Answer 4

• This is actually an assumption of ANCOVA, but people often get the logic wrong
  
  • People often think “well, if my CV differs across the levels of my IV, then my CV is a potential confound and I should control for it”
  • The logic sounds right, but it doesn’t really consider how ANCOVA works
  • For ANCOVA, we actually want our CV(s) to be independent from our IV(s); that is, our CV doesn’t differ across the levels of our IV
  • If this assumption is violated, it makes interpretation very tricky
    ○ Think multicollinearity in multiple linear regression
    It’s hard to know what effect the IV is having, and what effect the CV is having

Question 5

Mixed ANOVA vs ANCOVA

Answer 5

• As I’ve said in the past, statistics isn’t black and white, and you’ll potentially run into many situations where your data could fit into many different statistical tests
  
  • Usually, the choice will be based on your research question:
    ○ ANCOVA is usually for situations where you want to know whether the DV differs between groups, while trying to account for the effect of the CV on the DV
    ○ E.g., You want to know whether anxiety is lower after treatment, while controlling for what it was before treatment
  • Mixed ANOVA is for situations where you want to know whether the measures differ from one another
  • E.g., You want to know whether anxiety is lower at post-test compared to pre-test
  • However, this situation is even more ambiguous than Mixed ANOVA vs MANOVA
  • For pre/post test designs, Mixed ANOVA and ANCOVA are both commonly used
  • There’s no real consensus on which is best
  • However, each test has different assumptions, so sometimes people choose based on which test’s assumptions are/aren’t violated
  • However, in our alcohol example before, it feels like a pretty clear case for ANCOVA
  • Other situations are clearer cases for Mixed ANOVA
    ○ The example from the Simple WS ANOVA Lecture (Week 6 last semester)
    ○ People listen to Coldplay/Eminem/Whale and have their anxiety measured

Question 6

ANCOVA assumptions

Answer 6

• The standard ones:
  
  • DV (and CV) should be at the scale level
  • Data should be normally distributed
• The ones for whatever type of ANOVA you’re using:
  
  • Equal Variances for BS and Mixed ANOVA
  • Sphericity for WS and Mixed ANOVA (though can be avoided with LMEM)
  • Equality of covariance matrices for Mixed ANOVA (though can be avoided)
• Two ones unique to ANCOVA:
  
  • Independence between the IV and the CV
    Homogeneity of regression slopes

Question 7

Independence between the IV and the CV

Answer 7

• Essentially, we want the CV to be roughly equal across the levels of the IV
• We can run an ANOVA to ensure the covariate is independent of the IV
  
  • For this check, the covariate would be treated as the dependent variable rather than a covariate
  • If the ANOVA is non-significant, this suggests that the covariate is independent of the IV
  • Variance of the covariate is not explained by the independent variable
• If the ANOVA is significant, this may suggest that the covariate is not independent of the IV
  
  • Variance of the covariate is explained by the independent variable
• Essentially, we want the relationship between the CV and the DV to be roughly equal across the levels of the IV
  Essentially, we want there to be no interaction between the CV and the IV on the DV

Question 8

homogeneity of regression slopes

Answer 8

Homogeneity of regression slopes
- Essentially, we want the relationship between the CV and the DV to be roughly equal across the levels of the IV
- Essentially, we want there to be no interaction between the CV and the IV on the DV
- We can add the interaction term between the CV and IV into our ANCOVA
- Technically this isn’t “ANCOVA” anymore, but it is conceptually the same
- If the interaction is non-significant, this suggests that the assumption is met
- The relationship between the CV and the DV does not change across the IV
- If the interaction is significant, this suggests that the assumption is violated
The relationship between the CV and the DV does change across the IV