week 6 within subjects & mixed design ANOVA Flashcards
Repeated Measures ANOVA
(RM Anova)
-One way within-groups anova. The iv is a within subjects variable.Repeated measures taken for each subject. Used when tx/control measures are used on the same subject more than twice (if only twice, use t-test).
-overcomes between subject variability (error).
-if the effect is big, we will tend to see a homogenous change where most of the subjects are responding/improving at the same rate and time (eg over time, all subjects are improving with grades/healtth score etc. If not all occurring in similar fashion, is more likely an indicidual response as opposed to a tx response etc. (even though looking at repeated measures for an individual, stioll need to see the response over eg 100 indiviudals). If the effect is not consistent, will lose power.
-considers change over time for the same group of individuals.
-can possibly find an effect with a smaller sample size cf a between groups analysis, because there is less “noise” from variation as there is less variation within subjects.
Variance
Total Variance=between subjects variance + within subjects variance.
Within subject variance=treatment variance + how differently individuals behave over time (error).
Error here is also called Residuals.
F=treatment variation/error variation.
F=MS treatment/MS residual.
Degrees of freedom eg; eg 10 subjects measured 3 times;
Total= ((N of participants) x (N of groups)) -1=(10 x (3))-1=29
Between subjects =N of subjects-1=10-1=9
Within subjects=Total-between subjests=29-9=20
Treatment=N of groups -1=3-1=2.
Residual=(N of participants-1)x (N of treatments-1)=(10-1) x (3-1)=9x2=18
therefore degrees of freedom is F(2, 18)=F(treatment, residuals).
error
The Error term is the subject x treatment interaction.
It is the inconsistency of how different individuals respond to the different tx conditions. eg tiredness etc,
If the interaction is large, the treatment effect is not the same for all subjects, therfore we will have a larger error term, and a smaller F.
Consider how big is the effect, compared to individual variability?
Sphericity
Sphericity occurs when the tx is consistent for all participants. (All participants may not start and finish at the same score, but the change in their scores over time is consistent). Sphericity is akin to homogeneity of variance.
Assumptions for within-group Anova
1.Independenece of scores is assumed. But If you have 2 measures done too closely together, they are no longer independent. eg, might gain “practice experience”. Overcome this with study design.
2.The dv is assumed to be normally distributed in the population. But Anova is fairly robust to this being violated provided have large enough size and equal groups. Sometimes might remedy with transformations.
3.Homogeneity of co-variance (compound symmetry sphericity).There is going to be some variance between different time points of the same individual. The greater the time between measures, the greater the variability.
repeated measures anova example run through
The following are the steps of the analysis plan for RM ANOVA that were discussed in the video example:
1.GLM—repeated measures. (general linear model)
2.Planned contrasts (repeated/simple)—not post hoc here.
If select “repeated”, will test fore differences b/n the levels (within subject) in their sequential order. eg level of change between kindy and prep, prep and grade 1, etc.
If select “simple” will check for differences relative to a baseline eg kindy level of reading as baseline. (baseline will be the first item specified).
3.Run.
4.Inspect descriptives. (before run, request descriptives, estimates of effect size, observed power)(requesting plots often helps view patterns)
5.Check assumptions.
6.Inspect ANOVA result.
7.Check planned comparisons.
Mauchley’s test of sphericity is looking at differences between different kids. ie so assumption of sphericity is violated (kids vary between each other), therefore we can use several different corrected tests, but often correct using Greenhaus-geisser. with the corrections, the p values are corrected to make it less likely to find an effect. (to reduce risk of false positive). ie by applying a correction (more stringent test), some values which were significant may become insignificant. In this example, everything was still significant.
repeated measures anova 2
“since the data violated sphericity assumptions, we used a corredcted Greenhaus-geisser correction with our Anova. The effect of grade was significant; F(1.41, 46.16)=82.95, MS=46.45, p<.001”.
repeated measures anova3
Below shows the ouput for the within subjects contrasts (repeated).
Make sure check effect size as well as significance.(sometimes you might have something very significant but only a small effect, and sometimes not significant but a large effect….etc)
repeated measures anova4
The error bars in the below picture are somewhat senseless-they are showing the between-groups variance when the repeated-measures anova doesn’t use this!
Summary;
-Same participants used in all treatments(measures)
-No individual differences (variations due to individual subjects must be removed from the analysis (otherwise doing a between-groups anova)
-individual differences can be measured and subtracted.
-if there is a smaller error term, there will be a larger F value
-homogeneity of variance? what do if violated?
-run contrasts (simple or repeated).
Mixed design Anova
A mixed-design ANOVA combines two types of IVs in one analysis. Mixed design ANOVA can include:
a within-groups factor, e.g. the same child at different grades
a between-groups factor, e.g. low versus high socioeconomic status (SES).
However, the more IVs you have in your model, the harder it is to interpret. For example, it will be difficult to interpret what is happening with a four-way interaction.
We are interested in a mixed design for the interaction. Thus, we include a within-subjects factor and a between-subjects factor because we assume that something interesting is going on in that interaction. In the example of a within-groups factor of the same child at different grades and a between-groups factor of low versus high socioeconomic status (SES), we are not just interested in whether children change over time regarding reading scores. We’re also not just interested in whether there is a difference between low and high-SES kids regarding reading scores. The mixed design aims to investigate whether the change over time across different grades is different for low and high SES students.
See picture below;any time the lines are not parallel, there may be an interaction. The lines do not have to cross, for an interaction to occur. The larger the sample size, the slighter the deviation from parallel might be and still be found significant.
Assumptions of mixed design anova;
-normality of distributions
-homogeneity of variance
-homogeneity of covariance (sphericity)
-homogeneity of variance-covariance matrices (Box’s M)
Comparisons
-may be post-hoc or a priori.
Interaction
-Any significant interaction event requires further probing (this may be done with a priori hypoetheses) or can also;
-check pairwise comparisons (requires manual syntax)
-conduct a separate simple effects analysis.
Mixed design anova eg
eg I/v repeated measures kindy, prep, grade 1.D/v is reading score. now also with a between subjects factor of high and low SES level.
therefore a 3x2 mixed design anova. There is a within-subjects factor (grade, 3 levels) and a between -subjects factor (group, 2 levels).
This means what we need to find:
a) main effect (within subjects);grade-do reading scores change over time?
b)main effect (between subjects);SES-that is, is there a difference in reading score between low SES and high SES?
c)interaction (between x within). Is the change in reading score over time the same for high and low SES groups? The interaction is what we are most interested in.
If we ignore grade for the moment, and just consider SES, then we have 2 groups (high and low) and we are looking at the between-subjects main effect.
Then, if we ignore SES for the moment, we are looking at 3 groups (kindy, prep, grade 1). This is the within subjects main effect.
Bear in mind, that sometimes it does not make sense to report every main effect. eg Might not be meaningful to report a main effect of time, when what are really interested in is change in anxiety after tx etc.(depends on the research design).
testing mixed design anova outputs
testing mixed design anova output 2
box’s M
Box’s M;
-homogeneity of variance-covariance matrices. Similar to sphericity, but with a between-subjects factor. We want to have similar covariance levels for the repeated variables for each level of the between-groups factor.
-it is very sensitive for large data sizes p<.001. if Box’s M is violated, it means there is a lot more variability in 1 group than another.
testing mixed anova outputs3
We also do Levene’s test for homogeneity of variance as for a between’subjects anova, but with each group separately.
Also test Mauchleys, and if required, do a Greenhaus-geisser correction.
testing mixed anova outputs4
testing mixed anova output5
green is high ses and blue is low ses.
We found that the rate of learning in low ses children slows down from grade prep to grade 1, something that wasn’t obvious from the overall results. Why? Theory re crappy schools, lower level to begin with, less homework help?????etc etc.
testing mixed anova ouputs6
testing mixed anova outputs 7
testing mixed anova ouputs 8
Interpreted as if there were only 3 cells, ie all the groups not related to ses are simply averaged. Do children get better as they move from grade to grade? There was a main effect of grade, F(2, 64)=119.25, p<.001.
testing mixed anova ouputs9
violations of sphericity
What do you do with violations of sphericity? You can use multivariate tests. These are far less affected by violations of sphericity than the univariate tests. (You still need homogeneity of variance-covariance matrices, but the requirements are not so stringent).
You will lose power (i.e. it will be harder to find significance).
If both univariate and multivariate results are the same, report the univariate, but note that the multivariate solution is similar. If not, clean experimental designs are generally more appropriate for univariate analysis than non-experimental designs.
(Tabachnik & Fidell, 2013, p. 422)
Dr J’s mixed anova designs collab
OVERVIEW OF MIXED ANOVA;
-Anova with 2 or more i/v’s/factors, where at least 1 is b/n groups, and 1 is a repeated measures factor. (helps to spell out which is which).
-Depending on the approach used and the asummptions breached, the analysis can be completed within a single mixed design anova OR may require split-file follow up one-way anovas.
-follow up analyses within the mixed design anova ouput uses pooled variance (standard errors across groups and is the multivariate approach).
-asplit file approach allows you to do a follow up analysis where variances/standard errors are not pooled across groups and provides both traditional and multivariate approaches.
TRADITIONAL VS MULTIVARIATE APPROACHES TO REPEATED MEASURES FACTOR ASSESSMENTS;
- the assessment of repeated measures factors can be conducted via 2 approaches and SPSS provides BOTH within its ouput.(traditional approach and multivariate approach). They are calculated differently but the calculations are complex.
TRADITIONAL METHOD;
based on difference of scores b/n levels of the repeated measures factor. eg time 1, time2, times. Looks at the differences for each individual b/n time 1 & time2, b/n time 2&time3 etc (note USE TRADITIONAL FOR assignmet1b)
MULTIVARIATE METHOD;
Treats each repeated measure as a different dv (as if it has multivariate d/v’s). eg Considers time 1 as a dv, time 2 as a differnt dv etc.
TRADITIONAL/MULTIVARIATE
-the 2 methods lead to some different follow ups. Need to consistently follow the same path.
-SPSS will automatically report both, just report 1.
- The results will vary in exact F value and degrees of freedom but will usually agree on significance. (Rarely, where a results ison the cups of significance, might have the traditional method (has a fraction more power) reporting a significance and the multivatiate method reporting a non dignificance).
ASSUMPTIONS AND THEIR TESTS;
1. Normality
-normality tests (eg Kolmogorov-Smirnov/Shapiro Wilks etc. And do follow ups re skew and kurtosis if required.
-Normal distribution for each repeated measurement of the dv for each group (level of the b/n groups factor)
-run separately to your anova
2. Homogeneity of Variance
-Levene’s test. Similar variance across groups factor for each repeated measurement of the dv?
3. Spericity
-Mauchly’s test. Similar variance for sets of different scores b/n repeated measurements of the dv across groups (only for traditional approach). ie are all the differences in individual scores over time similarly varied across the different groups?
4. Homogeneity of Covariance
-Box’s M
-similar relationships/ covariances across groups of the between-groups factor b/n repeated measures of the dv. Co-variance is like the precursor to correlation. ie are 2 sets of scores going up/down etc together?
Dr J’s mixed anova designs collab2
an eg;
There has been a suggestion that varying the contrast of text & paper colour can aid reading. In particular might be helpful for children with dyslexia.
in our study, 20 children were given some txt passages to read and their reading speed noted.
Dv =reading speed in seconds.
B/n groups iv=dyslexia status (with & without)
Repeated measures iv=text paper combination;
white text on black paper
black text on white paper
black text on yellow paper
different texts. 10 children with dyslexia, 10 without.
BELOW SHOWS HOW TO BASICALLY REPORT. REmember that if there is a significance, go down a level. if there is significance at the drilled down level, it becomes the more deeply reported result, as to report an upper result in greater depth can be confusing because it will vary with the lower down effects.
RUNNING IT;
1. Run mixed anova and check all output
2.run one-way repeated measures anova using split files approach and check ouput
3.work out which bits to report and which ignore…
in data file 2=have dyslexia.1=do not.
each child has a reading score on different text/background.
> Analysis>general linear model>repeated measures
need to name the repeated variables measure (within subject factor name). Have named it Text_paper_mix. there are 3 levels. define which is level 1, 2, 3.Note how have labelled as sometimes ouput will only refer to as level 1 not black on white etc.
put dyslexia status as between subject factors
