Week 5 MANOVA From Laerd Website

Question 1

In what way is MANOVA an extension of ANOVA?

Answer 1

MANOVA is an extension of the one-way ANOVA that incorporates two or more dependent variables rather than just the one dependent variable

Question 2

How is MANOVA different to ANOVA?

Answer 2

ANOVA tests for differences in mean values between groups whereas MANOVA tests for the ‘linear composite’ or vector of the means between groups

Question 3

What is MANOVA, in essence?

Answer 3

Essentially, MANOVA combines the dependent variables to form a ‘new’ dependent variable in such a way as to maximize the differences between the groups of the independent variable. It is between this new, composite variable that you test for statistically significant differences between the groups.

Question 4

What is required to run a MANOVA?

Answer 4

One independent Variable that is categorical (2 or more groups - e.g. Transport type: bus, train, tram, pedestrian, car)
2 or more dependent variables that are continuous (e.g. satisfaction score, motivation score)

Question 5

What are the seven assumptions of MANOVA?

Answer 5

1. independence of observation
2. adequate sample size
3. no univariate or multivariate outliers
4. multivariate normality
5. Linear relationship
6. Homogeneity of the variance-covariance matrix
7. no Multicolinearity

Question 6

MANOVA assumption 1 is independence of observations. What does this mean?

Answer 6

Independence of observations means there is no relationship between the observations in each group or between the groups themselves.
e.g. there must be different participants in each group with no participant being in more than one group. This is more of a study design issue than something you can test for, but it is an important assumption of the one-way MANOVA.

Question 7

MANOVA assumption 2 is adequate sample size. What does this mean?

Answer 7

Although the larger your sample size, the better; for MANOVA, you need to have more cases in each group than the number of dependent variables you are analysing.

Question 8

MANOVA assumption 3 is no univariate or multivariate outliers. What does this mean?

Answer 8

There can be no (univariate) outliers in each group of the independent variable for any of the dependent variables. This is a similar assumption to the one-way ANOVA, but for each dependent variable that you have in your MANOVA analysis. Multivariate outliers are cases which have an unusual combination of scores on the dependent variables.

Question 9

MANOVA assumption 4 is multivariate normality. What does this mean & how can we test it?

Answer 9

This is an assumption that cannot be directly tested in SPSS. Instead, the normality in each group of the independent variable for each dependent variable is assessed.

Question 10

MANOVA assumption 4 cannot be tested directly in SPSS, so how do we test this assumption?

Answer 10

normality of each of the dependent variables for each of the groups of the independent variable is often used in its place as a best ‘guess’ as to whether there is multivariate normality.
i. e. If there is multivariate normality, there will be normally distributed data (residuals) for each of the groups of the independent variable for all the dependent variables. However, the opposite is not true; normally distributed group residuals do not guarantee multivariate normality.

Question 11

So, how do we test for Normality?

Answer 11

• Q-Q Plots if large sample size
• Shapiro-Wilk if smaller sample size & not confident interpreting Q-Q Plots
• Greater is good - sig of greater than .05 means normality assumption is met

Question 12

What do I do if my normality assumption is violated?

Answer 12

You can transform the DV(s) so that, hopefully, your transformed DV(s) is normally distributed.
If a DV is not normally distributed for any particular category of the IV(s), the DV needs to be transformed for all groups. You cannot just transform the data for one particular group without transforming the data of all the other groups (i.e., you have to transform every value of the DV).

Question 13

How do I transform my data if it is Positively skewed?

Answer 13

• Moderately positively skewed data = Square root transformation
• Strongly positively skewed = logarithmic transformation
• Extreme Positively Skewed = Inverse or reciprocal transformation

Question 14

How do I transform my data if it is Negatively skewed?

Answer 14

• Moderately negatively skewed data = reflect & square root transformation
• Strongly negatively skewed = reflect & logarithmic transformation
• Extreme Negatively skewed = Reflect and Inverse Transformation

Question 15

MANOVA assumption 5 is Linear relationship. What does this mean & how can we test it?

Answer 15

There is a linear relationship between the dv(s) for each group of the iv. If the relationship is not linear, it can lead to a loss of power to detect differences. This assumption can be tested with scatterplot matrices.

Question 16

MANOVA assumption 6 is Homogeneity of the variance-covariance matrix. How do we test this assumption?

Answer 16

Homogeneity of variance-covariance matrices can be tested using Box’s M test of equality of covariance

Question 17

MANOVA assumption 7 is no Multicollinearity. How do we test this assumption?

Answer 17

One of the most straightforward is to run correlations between the dependent variables to see if there are any relationships that are too strongly correlated.

Question 18

Okay, so my data has multicollinearity, what do I do?

Answer 18

1. Remove one of the DVs that is highly correlated
   * This is the normal thing to do
2. Combine the scores to achieve a new DV that is a combination of the 2 (often requires using PCA & is tricky)

Question 19

What do I do if I find my data violates the assumption of linearity?

Answer 19

1. Transform one or more DV
2. Remove the offending DV
3. Run the analysis anyway and expect a loss of power

Question 20

What do I do if I find I have outliers?

Answer 20

1. check whether I have made any data entry errors (i.e., simply keyed in any wrong values into SPSS). If any of your outliers are due to data entry errors, replace them with the correct values & re-run the tests of assumptions.
   * Any new values entered could still result in the outlier remaining an outlier, or could lead to other data points now being classified as outliers!

Question 21

Okay, I have checked for Data Errors and still have outliers, now what?

Answer 21

• consider whether they are measurement errors (e.g., equipment malfunction or out-of-range values).
• Treatment of Measurement errors: an out-of-range value can be replaced with the largest valid value (i.e. if scale goes to 100, then a value of 100) it is better than loosing the data

Question 22

Okay, I have established no data errors and no measurement errors, I still have outliers, now what?

Answer 22

These are genuine data points: options:

• keep the outlier
• remove the outlier

Question 23

How do I decide whether to keep or remove the outliers?

Answer 23

Removing the outlier requires providing information about that data point so that a reader can make an informed opinion about why you removed it and how it might have affected your results. It can also help dispel any accusations that you might have removed a data point just to make your results look better.

Question 24

Okay, so I want to keep the outliers, what do I do?

Answer 24

Keeping 3 choices:
1. modify the outlier by replacing the outlier’s value with one that is less extreme (e.g., it is common to use the next most extreme value that is not an outlier or a value slightly larger, in order to maintain the order of values);
2. transform the affected DV(s)
3. include the outlier in the analysis anyway, as you do not believe the outlier(s) will materially affect the result.
With respect to point two (2), transformation can be an option as it can lead to outliers being disproportionately affected (“reduced in size”) so that they are no longer classified as outliers. However, transformations are usually not warranted unless your data is not normally distributed.

Question 25

What is a multivariate outlier?

Answer 25

Multivariate outliers are data points that have an unusual combination of values on the DVs
*e.g. high neuroticism but low anxiety

Question 26

How do I check for multivariate outliers?

Answer 26

I use Mahalanobis Distance:

*the larger the value of Mahalanobis distance, the more unusual the data point

Question 27

What are the critical values for Mahalanobis Distance?

Answer 27

2 DV = 13.82
3 DV = 16.27
4 DV = 18.47
5 DV = 20.52

Question 28

What if I find I have multivariate outliers?

Answer 28

initially same as univariate: check for data and measurement errors.
*2 choices = Keep or remove the multivariate outliers

Question 29

So what do I do if I want to remove the multivariate outlier?

Answer 29

• Ensure I make this data treatment clear in my results

* Remember Andy Field and other experts are not fans of removing precious data!

Question 30

Okay, I have seen sense, I want to keep my multivariate outlier, what do I do to it?

Answer 30

1. transform the affected DV

2. Include the multivariate outliers anyway

Question 31

Why might I choose to keep the multivariate outlier in my analysis?

Answer 31

• MANOVA is somewhat robust to multivariate outliers
• best option is to run the analysis with the outlier and then re-run without the outlier and check if it makes a difference to the results

Question 32

What post-hoc test options do I have with a MANOVA?

Answer 32

• Tukey’s will test for equal variances assumed will produce post-hoc tests for the univariate ANOVA’s (similar to post-hoc t-tests)
• others include: LSD, Bonferroni, Sidak, Scheffe, Duncan’s, Hochberg’s GT2

Question 33

What does the Box’s Test of Equality of Covariance Matrices tell me about my data?

Answer 33

This tests the null hypothesis that the observed covariance matrices of the DV’s are equal across groups
*I am looking for a p value greater than .001 otherwise I have violated the assumption of homogeneity of covariance-variance

Question 34

If the assumption of homogeneity of covariance-variance is violated, what do I do?

Answer 34

• if I have equal sample sizes I can probably go ahead & not worry
• if my samples are different options include:
  1. Transform the data for 1 or more DV
  2. run the test but use Pillai’s Trace instead of Wilks’ Lambda as Pillai’s Trace is more robust

Question 35

Remind me, what does Levene’s Test for?

Answer 35

“Levene’s test of equality of error variances” tests the null hypothesis that the error variance of the DV is equal across groups
*Remember, as with most assumption tests = Greater is good over .05 shows assumption has not been violated

Question 36

What if my Levene’s test is violated?

Answer 36

• Transform DVs
• however, transformations are not always successful & SPSS does not offer a robust method
• Can continue with analysis but accept a lower alpha level for MANOVA and need to use different post hoc tests

Question 37

When it comes to sample size, what does MANOVA require?

Answer 37

There must be as many cases in each group of the IV as there are number of DVs as a bare minimum

Question 38

Which set of results do we look at when interpreting the Multivariate tests output?

Answer 38

*Pillai’s Trace (not the intercept line but the 2nd line that shows the IV)
*p needs to equal less than .05 for significance
NB: Wilks’ Lambda is more robust

Question 39

What should I do if my MANOVA is not statistically significant?

Answer 39

Simply report the findings and offer explanation as to the results
*No need to run further tests

Question 40

Exciting times, my MANOVA is statistically significant, what do I do now?

Answer 40

• Move on to check the results of the univariate ANOVA
• This is displayed in the “Tests of Between Subjects Effects” table
• I need to interpret the row that lists the IV & then the error term below the IV line (together I get my dfs from here)

Question 41

What information do I get from my Tukey HSD (Honestly Significant Difference) Table?

Answer 41

• This is my post hoc univariate ANOVA comparison table

* If assumption of homogeneity of variance is violated, I can use Games-Howell instead

Question 42

Other than writing up my results, how else can I display my findings?

Answer 42

• Produce graphs to visualise my data

- e.g. profile plots for each DV against the IV (IV on X axis; DV on Y axis)