Chapter 13: General Linear Model, Comparing Means Adjusted for other Predictors (ANCOVA) Flashcards
What is an extraneous variable?
any variable that could cause changes in the dependent variable other than the independent variable
two types: confounds and nuisance variables
What are confounds?
an extraneous variable that correlates with the independent variable
if groups are different in two ways, how can we tell which causes the difference in means?
threat to internal validity (ability to determine causation) because they could explain an observed treatment effect
What are nuisance variables?
an extraneous variable that does not correlate with the independent variable
positive: can still include statistically significant results, can compensate with larger sample size
negative: creates noise in the data that makes it harder to find the difference between groups
groups not necessarily different in two ways
not a threat to internal validity because they cannot explain an observed treatment effect; however they make finding treatment effects more difficult
no alt explanation
statistical validity threat: harder to find the observed effect, but can still make conclusions
What is an ANCOVA?
analysis of covariance
- technique to handle situations with extraneous variabes when comparing means
- used when there are differences in your baseline groups
- main focus is to compare means, additional predictors referred to as covariates
- examine what effect a predictor variable has, adjusting for the effect of the covariate. rather than predicting outcome from group means, we predict it from group means that have been adjusted for the effect of the covariates
Why do we include covariates?
1) to reduce within-group error variance: if we can attribute some of the unexplained variance in the model to other measured variabless (covariates), then we reduce the error variance, allowing us to assess more sensitively the difference between group means
2) elimination of confounds: if any variables are known to influence the outcome variable being measured (confound), then including them as covariates can remove these variables as potential explanations for the effect of interest
conceptual: moderation; extraneous
statistical: interaction, covariate
How do one way ANOVA, Nuisance variable, and confounds differ?
From total variances in the DV…
In one way Anovas, there is variance accounted for by the model and then unexplained variance, not accounted for by the model. We hope to see that the variability that can be accounted for by group means is high relative to unexplained variance.
- high variability = higher F = lower p value (model works well, groups are sig dif)
In nuisance variables, there is the variance accounted for by the model and then unexplained variance which also includes variance explained by the covariate (noise accounted for and removed)
- nuisance influences the D, correlation = 0
- SSm, SSr, MSm, MSr all smaller = higher F = lower p value
In confounds, there is variance accounted for by the model and unexplained variance, with variance explained by the covariate interlapping into both sections.
- variance explained by your treatment above and beyond the shared varance w/ the covariate
- use unexlained variance as a ruler
ANCOVA assumptions
1) independence of the covariate and treatment effect
2) homogeneity of regression slopes
if violated - bootstrap parameters and use post hoc tests (robust), like LSD, Tukey, Bonferroni, Sidak
independence of the covariate and treatment effect
- covariate completely independent of the treatment effect (no relationship) - easier to interpret, more powerful test
- if the extraneous variable/covariate is a nuisance variable, it is going to be a lot easier to interpret than if it was a confound
- can be avoided by randomizing people to tx conditions to make all groups equal on the extraneous variables, so all the extraneous variables are going to be nuisance variables
- if you have a confound, shared variance decreases the observed tx effect
- lack of indepdndence obscures a real tx effect, spurious effects occur
- interpretation of results harder if there is a confounding variable as your covariate, if there’s a relationship b/w covariate & tx
homogeneity of regression slopes
-the relationship b/w the covariate and the outcome variable is the same across all levels of the IV
- if violated, a more robust ANCOVA must be conducted
- if the assumption is met, the regression lines should look similar for each group of participants on the covariate
- if one line is different, the assumption is not met. even if there is only one group of three that’s not met (heterogeneity of regression slopes), it is still violated
Sidak correction
like Bonferroni post hoc test, but less conservative so it should be selected if you are concerned about the loss of power associated w/ bonferroni
to conduct an ANCOVA…
3 steps
1) run a regular ANOVA w/o the covariate (if it is not sig, no dif among the means when not including a covariate)
2) check to see if covariate is a nuisance or confound/independence of the covariate and tx effect (if it is not sig, there is no dif b/w the 3 groups on the covariate, which is good)
- if we don’t include the covariate in the model, we don’t find an effect, and there is no evidence that there’s differences across dif groups on the covariate, then it appears it is a nuisance not confound
3) run ANCOVA
testing the assumption of homogeneity of regression slopes
-relationship b/w the covariate/outcome consistent across all dif levels of the predictor
- we want no interactions b/w the covariate and the IV
- if there is a significant p value between the covariate and tx effect, this is not good and can lead to invalid conclusions. the assumption has been violated
what can we do if the assumption of homogeneity of regression slops is violated?
- robust ANCOVA
- estimate the difference b/w means at dif levels of the covariate
- bootstrap CIs - if they don’t contain 0, then theres a dif b/w means, which is what we want to see. we also want to see significant p values. if this doesn’t occur, there is no evidence that the groups are different from eachother, robust analysis didn’t find that X was effective
effect size measures
for main findings…
eta^2 = SSeffect/SStotal (ranges from 0-1.00, variance in DV that can be accounted for by covariate)
partial eta^2 = SSeffect/ SSeffect + SSresidual (proportion of variance that a variable explains that can account for by group membership that’s not also explained by other variables in the analysis)
for planned contrasts…
rcontrast = square root of t^2 / t^2 + df
