Week 4 ANCOVA Flashcards
to address the learning aims of week 4 lecture including: Assumptions of ANCOVA Why do we check for Homogeneity of Regression? Interpretation: Main effects & Covariates
When do we use ANCOVA?
- ANCOVA may be used as an extension to any type of ANOVA design, including MANOVA
- It is used when you want to statistically control for a covariate(s).
What is a Covariate?
- A covariate (CV) is a variable that must be related to the DV, which research suggests may impact on the findings.
- when we say control we are saying we are removing the effect of that variable (the CV) on the DV.
What does it mean to Statistically control for Covariates?
*when we say control we are saying we are removing the effect of that variable (the CV) on the DV.
Which is more Powerful ANOVA or ANCOVA?, and why?
*ANCOVA is more powerful than ANOVA because it removes the amount of error variance apportioned by the CV on the DV – think about this and remember the f-test statistic of ANOVA.
When & Why do we use ANCOVA?
ANCOVA can be used as part of:
- One way between groups ANOVA – One-way ANCOVA
- Two way between groups ANOVA – Two-way ANCOVA
- Multivariate ANOVA – MANCOVA
What does ANCOVA actually test for?
To test for differences between group means when we know that an extraneous variable affects the outcome variable. Used to control known extraneous variables.
*We use ANCOVA to test differences between group means when we know that another variable (CV) may impact on outcome variable (DV).
In what way are the principles of ANCOVA are quite similar to ANOVA
- ANOVA’s equation of the value of variance apportioned to the numerator (between subjects) divided by the value apportioned to the denominator (within subject effect).
- Removing error variance from the denominator will increase the overall value. Resulting equation’s value will be larger.
How is ANCOVA different from ANOVA?
*You can control for a known extraneous variable or in repeated measures, where you want to control for the baseline measure in a 2nd administration of a variable. *ANCOVA assesses change in the outcome with the removal of the impact of the baseline to see true differences.
What are the advantages of ANCOVA?
- ANCOVA reduces Error Variance
- By explaining some of the unexplained variance (SSR) the error variance in the model can reduce the within/ error variance (noise).
How does ANCOVA provide Greater Experimental Control?
By controlling known extraneous variables, we gain greater insight into the effect of the predictor variable(s).
Increases power to detect significant differences
How do you decide which Covariates to control for?
- decisions are derived from the theoretical and empirical literature
- I need to know my research area well, have a strong theoretical basis to argue for covariate use.
What properties should Covariates have?
- Covariates should be continuous variables that are measured reliably and need to be correlated significantly with the DV.
- Logically - this makes sense. Having a covariate that is not significantly correlated would do little in allowing adjustment of the DV to occur.
What points should I consider when using more than one Covariate?
- Each covariate should only share a moderate amount of variance
- Each should contribute uniquely to the variance
- They must be measured before the treatment or experimental manipulation, so the covariates are not influenced by the treatment . That is, you don’t unintentionally control for some of the treatment effect in the analysis.
What are the assumptions of ANCOVA?
- ANCOVA assumes covariates are measured reliably & are valid.
- Covariates should be suitable for the intended population
- Covariates should correlate well with the DV but not with each other.
- since overlapping covariates would contribute little to a reduction in error variance
- Should be a linear relationship between the DV and covariates as well as between covariates
What needs to happen with the Homogeneity of regression slopes?
- The relationship between covariate & DV for each group must be the same across groups.
- Unequal slopes indicate an interaction between covariate and treatment.
- adjustments are made on the DV according to the overall correlation with the CV. Different slopes across the groups would make the adjustments uninterpretable.
When is ANCOVA often used?
ANCOVA is often used to evaluate the impact of an intervention while controlling for pre-test scores
The example in the lecture introduced ANCOVA as a means of evaluating the impact of an intervention while controlling for pre-test scores. Can you remember the design?
Variables:
IV – Categorical - (2 groups Maths Building, Confidence Building)
*DV – 1 Continuous - Fear of Statistics after intervention
*Covariate (CV) - Baseline measure of Fear of Statistics prior to participating in the intervention groups
The IV was the intervention, what was that?
IV – Student Gp (1) receives maths skills intervention Student Gp (2) receives confidence building intervention
The DV was the fear of stats after the intervention (time 2), why was the covariate CV – Scores on Fear of Statistics – Test Time1?
By controlling the Scores on Fear of Statistics – Test Time1, i.e. reducing this to er for all participants, the effect on each intervention is clearly identifiable, without the confusion of the original fear of stats (the CV)
What would I do if my covariate and DV had a very weak correlation?
Reconsider, if it is not going to reduce error variance? What’s the point of using it?
How do I check for interactions between covariate and treatment?
Inspect scatterplot – are the 2 lines similar in their slope? Alternatively, we can also test this statistically if we write the syntax OR build terms in GLM
What does Hills suggest when requesting scatterplots?
- I can request a scatterplot for each group, and compare them
- I can look at the 2 groups simultaneously to see if there is a difference in slopes.
How do I test Homogeneity of Regression slopes?
either by graph or by building a custom model.
e.g. UNIANOVA fost2 BY group WITH fost1
/METHOD=SSTYPE(3) /INTERCEPT=INCLUDE
/CRITERIA=ALPHA(0.05) /DESIGN=group fost1 fost1*group
What would I do if the Assumption – Homogeneity of regression slopes is violated?
Consider again – not going to reduce error variance? So little point of using it.
When I have run the Homogeneity of Regression Assumption, what should I do?
NB: Remember to turn off custom in “model”, after you run this analysis or otherwise you will get very different results in the actual analysis
What am I looking for when I view the results of the Homogeneity of Regression slope test?
- I am only interested in the interaction when testing for homogeneity of regression slopes.
- I ask if the interaction significant at p<.05?
- If not, this is good & meets the assumption - otherwise if sig would indicate baseline (covariate) group differences existed (Bad)
So what does it mean when the assumption of Homogeneity of Regression is met?
*The slopes do not significantly deviate between the 2 groups.
A note from Field (pg 374): What is the difference between Tukey’s Least Significant Difference (LSD) & performing multiple t-tests on the data?
- The LSD pairwise comparison makes no attempt to control for the Type 1 error and is equivalent to performing multiple t-tests on the data
- LSD requires the overall ANOVA to be significant
- Tukey can be used when you have equal sample sizes & group variances are similar
What table do we look at to check for significance in the Covariate? & how does Field help us interpret this?
We use the “Tests of Between-Subjects Effects” table
- If the covariate is significant it means that the covariate significantly predicts the DV
- when the effect of the covariate is controlled (held at 0), we are then able to observe the effect of the IV on the DV WITHOUT the influence of the CV
What might I check for if I run an ANOVA with no covariate included in the analysis?
- Running an ANOVA first can inform me whether there is a significant difference between groups
- I can also determine whether the significance between groups is higher or lower without the covariate
What can I check for if I run the EM Means Output with the Covariate removed?
*I can check to see if there is a difference in means when the covariate is removed
When running a two-way ANCOVA (an additional IV), what will the “Tests of Between-Subjects Effects” output tell me?
*Whether there is significance in the separate IV’s, in the Covariate, & in the Interaction between the 2 IV’s
The output then produces 3 separate tables, one for each IV, & one for the interaction between IV’s - what is happening to the Covariate at this time?
The covariate is controlled (i.e. is set to zero for all participants)
*each table has a little sentence underneath it to indicate this: “a. evaluated at covariates appeared in the model (CV name) = — (value) for each IV and for the interaction:
“a. covariates appearing in the model are evaluated at the following values (CV name) = … Value
What can producing an “Estimated Marginal Means of (CV Name)” graph indicate?
The graph will depict the impact of IV on the other.
*If they cross over there is an interaction
How can ANOVA / ANCOVA be conceived as Regression?
*We can run a regression with Reading Score as the outcome variable (DV) and Training (Maths Skills or Confidence Building) as the predictor (IV), Note:
Intercept is the mean of the group coded as 0 (Maths Skills)
b for the Dummy Variable is the difference between the means of the Maths Skills or Confidence Building groups
*You could extend the regression example by including the covariate into the model at the 1st step as you would, if you used this in HMR (Hierarchical Multiple Regression).
ANCOVA shares the same assumptions as ANOVA, what are these?
- Data should be interval or ratio (Scale SPSS).
- Scores randomly sampled from the population.
- Scores are independent.
- Scores on the dependent variable are normally distributed.
- Homogeneity of Variance (Levene’s) needs to be met
What are the assumptions specific to ANCOVA?
*Covariate needs to be reliable and normally distributed.
*Needs to be linear relationship between covariate & DV for each group.
&Homogeneity of Regression across groups. There shouldn’t be a difference in the slopes across the groups. *The covariate is filtered across the entire sample and does not differentiate across groups.
What should one remember when checking normality assumptions?
- When checking for normality – don’t forget to do this for each group, by splitting the file by group prior to running the explore function.
- I also obtain z-scores while the file is split
- next assumption to rule out any possible outliers (values ±3.29 SD are seen as true outliers.
Just remind me, what do I use to check for normality?
- Kolmogorov-Smirnov or Shapiro-Wilk
* I don’t forget to run z scores for each group individually on the DV and the CV
What else can I do to check assumptions?
I can Check linear relationship between CV & DV and correlation between CV and DV should be significant. Homogeneity of regression obtained through scatterplot
Coming back to Homogeneity of Regression, what do I want to see when I look up my “test of between-subjects effects” table?
*I look at the interaction between the CV & IV.
*If they are not significant, the covariate may be used
(not significant means: there is no difference among the groups in their slopes)
What do the “descriptives” tables tell me about my conditions?
The descriptives show me the means prior to the covariate adjustment
What does the “Estimates” and “Pairwise Comparisons” tables tell me about my results?
- Estimates: The means after covariate adjustment
* Pairwise Comparisons: whether any interaction is significant after removing the effects of the covariate
What can we do to the alpha value when running a Bonferroni adjustment on Tukey’s LSD?
- start with alpha = .01 (in view of loss of power using small sample sizes)
- to avoid a type 2 error: divide the alpha level of .01 / 3 for each of the post hoc analysis
