6 manova

Question 1

What is MANOVA?

Answer 1

not a single outcome, but multiple outcomes

interactions between outcome variables

contrasting to see which groups differ

univariate = one outcome variable

multivarate = many outcome variables

Question 2

What is discriminant function analysis?

Answer 2

if you carry out multiple tests on the same data, type 1 errors mount up
→ shouldnt do separate linear analyses

whether groups differ along a combination of dimensions (interrelations are accounted for)

Question 3

What can be said about the power of MANOVA?

Answer 3

greater power than ANOVA in theory

however, contradictory

some studies show: diminishing power as correlation between outcome variables increases

others show that power with high correlations between outcome variables is higher

if you are expecting to find a large effect → MANOVA
measures need to be somewhat different
group differences are same direction for each measure

e.g. if one group has large difference, the other has no difference
→ power will be increased if those are highly correlated

Question 4

What is a threat of MANOVA?

Answer 4

choosing outcomes

you need a good theoretical basis

p-hacking → do hundreds of combinations and only choose the relevant ones

Question 5

For all subsequent cards, consider this example of a MANOVA.

Answer 5

OCD - intrusive images/thoughts

CBT on OCD

three groups:
1. CBT
2. behaviour therapy
3. no treatment

two outcomes:
1. occurence of behaviours
2. occurrence of cognitions

Question 6

What are matrices?

Answer 6

matrix = grid of numbers arranged in columns and rows

specify its dimensions using numbers

values within a matrix = components or elements

columns and rows = vectors

square matrix = equal number of c and r

identity matrix
diagonal elements are 1
off-diagonal elements are 0

spreadsheet

Question 7

What is the statistical computation of MANOVA considering matrices?

Answer 7

we are interested in computing an F statistic, that represents how much variance can be explained by the fact that certain scores appear in certain groups

here: multivariate

sum of squares due to model variable

→ sum of squares and cross-products matrices

matrix that represents the systematic variance = H
hypothesis sum of squares and cross-products matrix
hypothesis SSCP

unsystematic variance = E
error SSCP

total amount of variance = T
total SSCP

Question 8

What is a cross product?

Answer 8

cross product = total value for the combined error between two variables

Example: If you’re looking at study time and test scores, the cross product will tell you if students who study more than the average tend to score above average and if students who study less than the average tend to score below average, indicating a relationship between study time and test scores.

Question 9

What is the sum of squares?

Answer 9

sum of squares = total squared difference between the observed balues and the mean value

Example: For test scores alone, the sum of squares tells you how much students’ scores vary around the average score. If all students scored the same, the sum of squares would be zero, indicating no variability.

Question 10

What are key differences between cross product and sum squares?

Answer 10

• The cross product is about the relationship and combined variability between two different variables, assessing how they co-vary.
• The sum of squares focuses on the variability within a single variable, measuring how much the observations deviate from the mean of that variable.

Question 11

How is univariate F for one outcome calculated?

Answer 11

three sums of squares

• within the outcome SSt
  
  • difference between each of the scores and the mean, sqaring them, adding them
  • or variance for action scores and multiply this by df (29)
• explained by model SSm
  
  • difference of each mean and the grand mean, sqaring it, multiplying the number of scores in the group, adding the values together
• explained by error SSr
  
  • taking difference between each score and the mean of the group
    square, add them
  • or multiply each group variance by N-1 and add

calculate the average sum of squares
→ divide the SS by their degrees of freedom

F statistic is the mean squares for the model divided by the mean squares for the error

Question 12

How are cross products calculated?

Answer 12

calculate each difference between mean and scores for each group

multiply these differences
D1 x D2

model cross product = multiply above with the number of observations

Question 13

What is the difference between grand mean and group mean?

Answer 13

grand mean = average of all data points

group mean = average within a specific group

Question 14

What is the total SSCP matrix?

Answer 14

The total SSCP matrix, T, contains the total sums of squares for each outcome variable and the total cross-product between the outcome variables. You can think of the first column and first row as representing one outcome variable and the second column and row as representing the second outcome variable

             actions    thoughts actions         SSt            CPt thoughts     CPt             SSt

Question 15

Are matrices additive?

Answer 15

Yes!
you can do
T = H + E

Question 16

What is HE-1?

Answer 16

univariate F is ratio of systematic variance to unsystematic variance

SSm/SSr

divide matrix H by matrix E

H - model

E - error

→ multiply by the inverse of a matrix

mutliply H by inverse E (E-1)

tricky to calculate so do not bother

HE-1 is conceptually the same as the univariate F statistic

Question 17

What are the main principles of discriminant function variates?

Answer 17

1. predicting an independent variable = predict a categorical variable based on a set of outcome variables, which are typically continuous
2. variate or component = linear combinations of the outcome variables
3. discriminant functions = distinguish between groups

e.g. Let’s say you’re studying the effectiveness of different therapies on depression and collect data on patients’ improvement across various metrics. Discriminant analysis would allow you to create a model that can predict, based on these improvements, which therapy a future patient might find most beneficial.

Question 18

What is the concept of maximisation in the context of discriminant function analysis?

Answer 18

maximisation

first discriminant function V1 is the linear combination of outcome variables maximizing the difference between groups

ratio SSm/SSR will be maximized for this first variate, but smaller values in subsequent variates

Question 19

What is the equation used in discriminant function analysis?

Answer 19

yi = b0 + b1X1i + b2X2i

V1i = b0 + b1Outcome11i + b2Outcome22i

b values are weights - obtained from eigenvectors of HE-1

eigenvectors are the vectors associated with a give matrix, unchanged by transformation of that matrix to a diagonal matrix

→ fewer values

Question 20

What common test statistics exist for MANOVA?

Answer 20

• pillai-barlett trace
• Hotelling´s T2
• Wilks´s lambda
• Roy´s largest root

Question 21

What is the pillai-barlett trace?

Answer 21

The Pillai-Bartlett trace is considered robust and is particularly recommended when the assumptions of MANOVA are not strictly met. It is less affected by departures from multivariate normality and homogeneity of variance-covariance matrices compared to other MANOVA test statistics.

sum of rations of each eigenvalue

Question 22

How should pillari-bartlett trace be interpreted?

Answer 22

• Values of the Pillai-Bartlett trace range from 0 to the number of dependent variables or the number of groups minus one, whichever is smaller.
• Larger values indicate a greater difference in the group means on the dependent variables.
• The significance of the Pillai-Bartlett trace is typically assessed via an F-test, where the trace is compared against an F-distribution to determine if the observed differences in group means could likely be due to chance.

Question 23

What is Hotelling´s T2?

Answer 23

hotelling-lawley trace

sum of eigenvalues for each variate

compares directly to univariate F statistic

Question 24

How should Hotelling´s T2 be interpreted?

Answer 24

• Larger values of Hotelling’s Trace indicate greater differences among the group means.
• Like Wilks’ Lambda, Hotelling’s Trace is converted to an F-statistic to assess significance.

Question 25

What is Wilks´s lambda?

Answer 25

product of unexplained variance of each of the variates ratio of error variance to toal variance

Question 26

How should Wilks´s lambda be interpreted?

Answer 26

- Wilks' Lambda values range from 0 to 1. - A value close to 0 indicates that the group means are very different in the multivariate space. - The significance is usually assessed by converting Λ to an F-statistic to determine if the observed group differences are unlikely to have occurred by chance.

Question 27

What is Roy´s largest root?

Answer 27

eigenvalue (root) for the first variate in a sense, same as Hotelling-Lawley trace but for the first variance only maximum possible between-group difference

Question 28

How should Roy´s largest root be interpreted?

Answer 28

- Roy's Largest Root examines the maximum variance explained by any linear combination of the dependent variables relative to the error variance. - A larger root suggests more substantial differences among groups on the most discriminant linear combination of dependent variables. - Its significance is also tested using an F-distribution.

Question 29

How can the test statistics be compared?

Answer 29

- Wilks' Lambda looks at the overall model variance explained. - Hotelling's Trace sums the individual discriminant functions' variance explained. - Roy's Largest Root focuses on the single most discriminant function.

Question 30

What are some assumptions MANOVA makes?

Answer 30

- **Independence:** Residuals should be statistically independent. - **Random sampling:** Data should be randomly sampled from the population of interest and measured at an interval level. - **Multivariate normality:** In univariate models we assume that our residuals are normally distributed. In the case of MANOVA, we assume that the residuals have multivariate normality. - **Homogeneity of covariance matrices:** In univariate models, it is assumed that the variances in each group are roughly equal (homogeneity of variance). In MANOVA we assume that this is true for each outcome variable, but also that the correlation between any two outcome variables is the same in all groups. This assumption is examined by testing whether the population variance covariance matrices of the different groups in the analysis are equal.

Question 31

What are suggestions for assumption testing?

Answer 31

chack the assumption of univariate normality of residuals for each outcome variable in turn Hotelling´s T2 is robust in the two-group situation when sample sizes are equal can be tested using Box´s test → non-significant if homogeneity especially if sample size is not equal → test it! Tabachnick and Fidell (2012) suggest that if the larger samples produce greater variances and covariances then the probability values will be conservative (and so significant findings can be trusted). However, if it is the smaller samples that produce the larger variances and covariances then the probability values will be liberal and so significant differences should be treated with caution (although non-significant effects can be trusted).

Question 32

How should a test statistic be chosen?

Answer 32

(1) for small and moderate sample sizes the four statistics differ little; (2) if group differences are concentrated on the first variate Roy's statistic should have the most power (because it takes account of only that first variate) followed by Hotelling's trace, Wilks's lambda and Pillai's trace; (3) when groups differ along more than one variate, this power order is reversed (i.e., Pillai's trace is most powerful and Roy's root is least); (4) unless sample sizes are large it's probably wise to use fewer than 10 outcome variables.

Question 33

What can be said about the robustness of the test statistics?

Answer 33

- all four are relatively robust to violations of multivariate normality, even though Roy´s root is affected by platykurtic distributions (flatter peak than normal distribution) - Roy´s root is not robust when homogeneity of covariance matrix assumption is untenable - when sample size is equal: Pillai-Barlett trace - with unequal group sizes, check the homogeneity of covariance matrices; if they seem homogeneous and if the assumption of multivariate normality is tenable, then assume that Pillai's trace is accurate.

Question 34

When should a follow-up analysis after a significant MANOVA be done?

Answer 34

follow a significant MANOVA with seperate ANOVAs some argue that univariate F statistics are “protected” by initial MANOVA → no inflated Type 1 error initial test is non-significant - subsequent Fs are ignored because any significant F must by a Type 1 error argument applies only to outcomes variables for which true group differences exist Bonferroni correction univariate Fs make no sense as follow up discriminant analysis!!

Question 35

What is the general procedure for a MANOVA?

Answer 35

explore data - check for outliers - boxplots, histograms, descriptives correct outliers/normality problems run the MANOVA - look at univariate F if you must discriminant function analysis

Question 36

Why are descriptives vital for MANOVA?

Answer 36

At the very beginning, descriptive statistics can provide an insight into any patterns in the data and set expectations for what may appear in the inferential statistics. At the very end, descriptive statistics are needed to frame the inferential statistics, that is, if the post-hoc test tells you that the means for condition A and B differ significantly, then descriptive statistics will inform you which condition has the higher mean.

Question 37

What is post hoc analysis?

Answer 37

compare each group to all others - There are many different types of post-hoc analyses – on this course, we recommend that you use the more conservative Bonferroni post-hoc test. Bonferroni controls for multiple comparisons. In the post-hoc comparisons table you need to consider all statistics but mainly the SPSS column titled ‘Sig’.

Question 38

How should the Box´s Test of Equality of Covariance Matrices be interpreted?

Answer 38

- should be non-significant - meaning covariates are roughly equal - Box’s M is notoriously susceptible to a range of factors and therefore is not the most reliable statistic. One rule is to use an alpha threshold of >.005 as opposed to >.05 to control for some of the issues with Box’s M; others simply ignore Box’s M and choose a different Multivariate statistic to compensate for any potential issues.

Question 39

How should Barlett´s Test of Sphericity be interpreted?

Answer 39

Bartlett: - only useful in univariate repeated-measures designs - ignore it

Question 40

What is important to consider in the MANOVA test statistics?

Answer 40

GROUP - all should reach significance - interesting, because if one does not, it determines the interpretation - useless to have all-or-nothing criterion - additional power associated with Roy´s - robustness of Pillai´s - → tells us nothing about which groups differed from which - BUT conclusion: type of therapy had significant effect on OCD - Each test statistic has its own strengths and weaknesses; however, for this course, it is recommended that when running a MANOVA, you report the Pillai’s Trace statistics. They are slightly less powerful than the other statistics that are available, but they are also less susceptible to violations of the MANOVA assumptions (e.g. significant Box’s M). Crucially, the MANOVA test statistics only tell you if a difference exists in your data, they do not tell you where or what the difference is. To find out the ‘what’ and ‘where’, you need to consider univariate test statistics.

Question 41

How should the Levene´s TEst of Equality of Error Variances be interpreted?

Answer 41

- should be non-significant - cus it strengthens the assumption of homogeneity of variance

Question 42

How should the univariate "test between-subjects effects" be interpreted?

Answer 42

- consider group and error generally!! - group = model sum squares - error = residual sum squares - corrected total = total sums of squares - identical to ANOVA → only hypothetical protection of inflated type 1 error - odd: the multivariate test statistics led us to conclude that therapy had had a significant impact on OCD, yet the univariate results indicate that therapy has not been successful - → suggests that its not either thoughts or actions that distinguish the groups, but some combination of them When the MANOVA is significant you should consider the univariate statistics for each dependent variable. These will help you to further narrow down the location of the significant effect in the data.

Question 43

What is the variance-covariance matrix?

Answer 43

covariance = average cross-product variance = average SS variance-covariance matrix = average form of the SSCP matrix correlation = standardised version of the covariance

Question 44

Give a general overview of MANOVA.

Answer 44

- MANOVA is used to test the difference between groups across several outcome variables/ outcomes simultaneously. - Box's test looks at the assumption of equal covariance matrices. This test can be ignored when sample sizes are equal because when they are, some MANOVA test statistics are robust to violations of this assumption. If group sizes differ this test should be inspected. If the value of Sig. is less than 0.001 then the results of the analysis should not be trusted (see Section 17.7.1). - The table labelled Multivariate Tests gives us four test statistics (Pillai's trace, Wilks's lambda, Hotelling's trace and Roy's largest root). I recommend using Pillai 's trace. If the value of Sig. for this statistic is less than 0.05 then the groups differ significantly with respect to a linear combination of the outcome variables. - Univariate F-statistics can be used to follow up the MANOVA (a different F-statistic for each outcome variable). The results of these are listed in the table entitled Tests of Between-Subjects Effects. These F-statistics can in turn be followed up using contrasts. Personally I recommend discriminant function analysis over this approach.

Question 45

When should planned comparison be done?

Answer 45

perfect for when you have specific hypothesis, clear baseline condition if only two levels of independent variable -> looking at the means of each level of IV if no hypothesis -> post-hoc comparisons

Question 46

When is MANOVA most appropriate to use?

Answer 46

When looking for differences between conditions with multiple related dependent variables in the research design. NOT prediction single IV/single DV mediation/moderation

Question 47

What is the correct order of MANOVA analysis?

Answer 47

Box’s M > Multivariate > Univariate ANOVAs > Post Hocs > Descriptive statistics 

Question 48

How should MANOVA be interpreted in SPSS?

Answer 48

- descriptive statistics - what appears to be going on - significant mean differences? - what tendencies can be observed? - box M - quite unreliable - not ideal if its significant, but it gets slightly ignored - multivariate results - pillais trace - use Roys of Box is largely unsignificant - ignore the intercept - univariate results - test of between-subject effects - each dependent variable seperately - look at the group row → p-values - post-hoc tests - if our MANOVA is sigificant, we need to find out where that significance lies - theres a bunch of mirror images - between group comparison, within group comparisons - is there a difference in outcome variables within the different independent variables? - double check with descriptives

Question 49

What is the difference between t-testing, ANOVA, and MANOVA?

Answer 49

key distiction: number of outcome variables t-test and ANOVA t-test = does a difference exist between two means? independent variable with two levels one dependent variable ANOVA independent variable with multiple levels one dependent variable