Profile analysis Flashcards
Advantages and disadvantages of profile analysis
+ Reduced error (within-group) variance e.g. participants function as their own control.
+ As a result it gives you greater statistical power than between-groups designs (fewer participants required)
- Order effects (e.g. first measure gives participant practice and changes their ability on the second measure if doing over multiple time points)
- Carryover effects- how they responded on last measure might affect how they decide to respond on next measure.
- Sensitization (sensitized to treatment).
What are segments in profile analysis?
- The analysis is done on the difference scores/segments (between two time points or between two questionnaires e.g. compare the difference between anxiety and depression or the difference between anxiety at time point 1 and time point 2 from time point 2 to time point 3 etc… Do the groups differ as a function of the time point
- The difference scores between levels of the RM IV are the dependent variable in profile analysis
- However when checking limitations/assumptions for this analysis individual values rather than the segments, of the levels of IVs are examined e.g. you wouldn’t test for normality on different scores.
What is the flatness hypothesis?
Refers to the main effect of the RM variable.
o Do all of the DV’s elicit the same average scores?
o If main effects are significant, this means the average scores are not the same for each of the dependent variables, so the profile is not flat. The flatness hypothesis is rejected.
o If the main effect of RM variable if non sig, the profile is flat. Flatness hypothesis accepted
o This is evaluated independently for each group, so the flatness is the slope of each line from one point of the graph to another along the horizontal axis.
o Typically this is only relevant when the profiles are not parallel: if they are not parallel, then at least one line is necessarily not flat. HOWEVER, sometimes profiles may be non-parallel but none may deviate significantly from flatness. So you can have a significant interaction and no significant main effect of the within-subjects variable.
What is the parallelism hypothesis?
Refers to the interaction.
o You need to report the multivariate statistics for this interaction
o No interaction means that the profiles were parallel, parallelism hypothesis accepted.
o This means that either the groups scored almost the same, or that one group scored uniformly higher than the other group on all levels of the DV.
o If there is a significant interaction, then the profiles are not parallel. The parallelism hypothesis is rejected.
o If there is a group-by-variable interaction: you then need to clarify the source of the interaction (with comparisons and contrasts).
What is the levels hypothesis?
Refers to the main effect of the between-subjects variable.
o Refers to the main effect of the between-subjects independent variable, or ‘grouping’ variable
o The test is basically asking does one group score higher on average across all measures or time points? (difference in levels refers to the combined scores for all levels of the DV, all of the time points or DV’s are collapsed into a group mean.)
If significant, profiles are not level. There is a significant difference in levels.
o The levels test produces a standard ANOVA table
o If significant, this means the groups have unequal levels, i.e. there is a significant main effect of the between subjects variable.
o If non-sig but profile was not parallel then the difference between groups (on the repeated measures variable) can be considered as being due to sampling error (?)
Importance of linearity assumption in profile analysis
For the parallelism and flatness tests, linear relationships among DVs is assumed (check by bivariate scatterplots)
o Major consequence is the loss of power in the parallelism test
o This violation can be circumvented by large sample sizes (large sample size in profile analysis most of these problems can be avoided)
What is homogeneity of the variance-covariance matrices?
Covariance is the unstandardised correlation between one variable and another.
Covariance among DV’s should be similar across groups. If the variances are not similar this can influence error variance, which is a problem as error variance is critical for estimating F.
Box’s test of equality of covariance matrices tests whether the covariance among dependent are similar across groups (whether the covariances within one group are similar to the covariances within another group) If significant, assumption is not met. Need to use p< .001 for rejecting this assumption because the test is very sensitive.
When do we need Box’s test? And what do we do if it is violated?
Box’s m is sensitive to large data files, meaning that when there are a large number of cases, it can detect even small departures from homogeneity. Moreover, it can be sensitive to departures from the assumption of normality.
o Box’s test is no longer necessary if samples sizes in each group are equal or near equal, it can be disregarded.
o However if samples sizes differ considerably and p
When does profile analysis have greater power than RM ANOVA?
• Profile analysis has greater power than univariate (RM ANOVA) when the assumption of sphericity (variability of differences between levels of the DV should be similar) is violated. When sphericity is violated profile analysis is more robust.
What is the sphericity assumption?
Sphericity refers to sphericity of the variance-covariance matrix.
Can be thought of as an extension of the homogeneity of variance in an ANOVA, and is a less restrictive form of compound symmetry (-there was an old exam model answer where they gave an extra mark for saying the words compound symmetry. However, this doesn’t appear in Devins slides).
Sphericity has 2 meanings
1. All pairs of the within subjects variables should have equal correlations (time 1 and time 2 correlation should equal time 2 and time 3 correlation etc.)
2. The differences between all pairs of the within subject variables should have equal variances (sphericity of the covariance matrix).
**note this is different to Box’s m which tests whether the matrices of different groups are equal.
If the sphericity assumption is not met, then the F value is positively biased (type 1 error rate is increased- falsely rejecting the null hypothesis too often).
For repeated measures designs this is likely to be violated, especially when time is the within subjects factof, because things measured more closely in time tend to be more highly correlated (i.e. correlation between IQ at 10 and IQ at 11, will be greater than the correlation between IQ at 10 and IQ at 13).
How do we test for sphericity?
Mauchly’s test in SPSS
If it is significant then the assumption is violated. If it’s non-significant then the assumption is met.
Mauchly’s can fail to detect departures from sphericity in small samples
If N is large, Mauchly’s has too much power (potential to violate the assumption when it shouldn’t be) – may want to consider using a more conservative alpha.
How is sphericity estimated?
Departure from sphericity is reflected in a parameter, epsilon
If E = 1, then there is perfect sphericity, as it falls below 1 the data increasingly moves away from sphericity.
Sphericity only applies where there are more than 2 levels on the within subjects IV
o Conceptually this makes sense because if you only have one pair of within subjects measures, then you only have one covariance, and no other covariances that it is required to be homogenous with.
o This can also be demonstrated mathematically when we apply the equation for the lower bound of epsilon.
o E has a ‘lower bound’ of 1/(k-1), where k is the number of levels in an RM factor
o Thus, if k = 2, then E = 1/(1-1) = 1, so with 2 levels there is always perfect sphericity so it doesn’t apply.
o If k=3, then E = 1/(3-1) = .5, so sphericity applies.
What can be done if the sphericity assumption isn’t met?
There are corrections available for the univariate RM ANOVA, i.e. Greenhouse-Geisser and Huynh Feldt. Both attempt to estimate E and use their sphericity estimates to correct DF for the F-distribution. The F statistic itself does not change, but as a result larger critical values are used (i.e. p values increase). This is to counteract the fact that when the assumption of sphericity is violated, there is an increase in type 1 errors due to critical values being too small.
Or, if the assumption of sphericity is not met, then we can treat the RM effect using RM MANOVA of profile analysis, which is more robust against sphericity violations.
Sample size considerations in profile analysis
o You must have more participants in the smallest group than number of DVS. E.g. you cant have 3 participants in anxiety group and 4 DVs. Do a quick calculation in the exam, see how many DVs there are and then say how many participants there needs to be in the smallest groups.
o (DV’s could potentially outnumber participants in longitudinal study with lots of DVs across time points over so many years)
o You need a large sample to achieve a good level of statistical power
o And because it is impossible to to evaluate the assumption of homogeneity of variance-covariance matrices when participant no is smaller than DVs
o If there are more DVs than participants in smallest group then you use RM ANOVA.
• Unequal sample size is not a problem in profile analysis (except with 2 or more between-group IVs), but may affect homogeneity of variance assumption. Unequal sample size may be an issue for ANOVA in a variety of more ways.
When should we follow profile analysis with contrasts?
o Needed when complex significant effects revealed by a profile analysis
o Complex means:
Only when main effects have >2 levels
All interactions, esp. those involving factors with >2 levels
o Usual issues apply: -protection of error rates for multiple comparisons -a priori vs. post hoc -choice of available methods
