Week 7 - Analysis of Variance Models MIP

Question 1

what is an analysis of variance?

Answer 1

Analysis of Variance: DV is continuous, IV(s) are categorical

• One-way ANOVA – 1 IV two levels (same as t-test)
• Two-way ANOVA – 2 IVs with multiple levels
• Between- groups, within-groups, mixed

Question 2

what is ANCOVA?

Answer 2

Combination of ANOVA and regression

• Three types of variables: DV, IV, and Covariate
• DV = continuous, IV = categorical, Covariate – (usually) continuous like regression

Question 3

how to choose covariates for an ANCOVA?

Answer 3

1. Covariates are usually continuous variables (although gender is sometimes used as covariate).
2. They are not variables of major concern in your research. If they are of major concern, they should be elevated to the status of an IV, and given proper attention in your introduction.
3. Covariates and the DV must be correlated. If not, there is little benefit for running an ANCOVA.
4. If multiple covariates are considered, they should be uncorrelated with each other. Highly correlated covariates add little, while consuming degrees of freedom; they may make the results unstable.
5. A covariate must correlate with the DV in similar ways in different experimental conditions - Assumption of Homogeneity of Regression.

• See section 5.1 and 5.2 of T&F(2014) for detailed issues

Question 4

when do you use ANCOVA?

Answer 4

1. In experimental designs where cases are randomly assigned to different conditions.
2. In quasi-experimental designs where random assignment is not possible.

Question 5

EXAMPLE: ANCOVA as an experimental design

Answer 5

A researcher wishes to examine the relative effectiveness of three counselling methods in improving clients’ self-esteem. Twenty-four clients attending a counselling centre were randomly assigned to one of three counselling methods (X). After several weeks, their levels of self-esteem were measured (Y). As previous education level varied widely across participants (from 7 years to 16 years) and was believed to correlate with self-esteem, it was used as a covariate (C).

Question 6

EXAMPLE: ANCOVA as a quasi experiment

Answer 6

A researcher wishes to examine the relative effectiveness of three counselling methods in improving self-esteem. Twenty-four adults attending a counselling centre assigned themselves, as convenient, to one of three counselling methods (X). After several weeks, participants’ level of self-esteem was assessed (Y). The non-random assignment led the three groups to have different levels of education. As self-esteem was believed to correlate with education level, it was used as a covariate (C).

Question 7

ANCOVA using adjusted means

Answer 7

ANCOVA adjusts the DV scores by using four pieces of information: (i) the initial DV score; (ii) the relationship between covariate and DV; (iii) the individual’s covariate value; and, (iv) the mean covariate value for the sample.

It is very important to understand that statistically controlling for pre-existing group differences in quasi-experiments cannot establish causal relationships as can be done with true experiments

Question 8

what are the assumptions of an ANCOVA?

Answer 8

1. ANOVA assumptions (independence of observations, normal distribution of errors, homogeneity of variance of errors), PLUS
2. Linear relationship between DV and covariate.
3. Homogeneity of regression – Covariate and DV relate in similar manner in all conditions; that is, regression lines for different conditions are parallel. This assumption is required because the average regression weight is used to adjusted DV for all cases.

Question 9

TRUE OR FALSE:
ANCOVA is useful in quasi experiment and correlational studies that involve many potential confounding factors. It is not relevant to true experiments where random assignment of participants is used.

Answer 9

False

Question 10

TRUE OR FALSE:
When ANCOVA is performed with the SPSS or JAMOVI, you first need to calculate the adjusted DV scores by using covariate, before running the ANOVA program.

Answer 10

False

Question 11

TRUE OR FALSE:

In ANCOVA, covariates are usually continuous variables

Answer 11

True

Question 12

What is a MANOVA?

Answer 12

Multivariate Analysis of Variance

MANOVA is an extension of ANOVA to where you have multiple DVs.

• A set of DVs are used together to test if there are group differences.
• To do this, the program computes a new variable, a linear combination of DVs (called discriminant function) that effectively distinguishes the groups against each other. Using the composite measure(s), a multivariate F-test is used to conclude on group differences.

Question 13

what are the advantages of using a MANOVA?

Answer 13

1. Different measures may have different weaknesses as well as strengths, and thus using them simultaneously can cancel out those errors.
2. Testing a hypothesis multiple times by using separate DVs increases Type I error (rejecting a true null hypothesis, or a false positive). MANOVA can provide a protection against this, only if the multivariate effect is interpreted.

Question 14

when do you conduct a MANOVA?

Answer 14

1. When you DVs are chosen carefully to tap important aspects of the target construct, as well as the scales having a good reliably.
2. When DVs are correlated moderately (this is slightly contentious *).
3. Equal cell sizes: it is important to have equal cell sizes.
4. Absence of outliers. MANOVA is sensitive to outliers, both univariate and multivariate. It is recommended that outliers are checked within each condition (cell).

Question 15

what is the power on a MANOVA?

Answer 15

Are correlated DVs desired for MANOVA?

• DVs should be uncorrelated in MANOVA (T&F, 2013)
• Relationship between the power of MANOVA and correlations among DVs depends on the effect sizes (of IV on DV) and the direction of correlation between the DVs. Thus, it seems wise not to assume that moderate correlation equals high power (Cole et al., 1994)

So what’s the case?

Question 16

4 pieces of advice when using a MANOVA?

Answer 16

1. When selecting outcome measures on which you expect sizable effect size (d = .5), MANOVA is more powerful if the measures are different (even negatively correlated) than similar (positively correlated), provided that the effects of IV on the DVs are in the same direction.
2. Assuming you have two potential DVs to use in MANOVA, one being much stronger (larger effect size) than the other (e.g., d = .5 and d = 0), MANOVA increases its power as the correlation between the two DVs (including negative r) increases.
3. When one seeks additional weak DVs to use in MANOVA, power increases most dramatically when the weak DVs are negatively related to each other.
4. Increasing the reliability of the DVs will increase the power of MANOVA. This makes good sense.

Question 17

what to do next if MANOVA multivariate F is significance?

Answer 17

Move onto univariate statistics next?
• BUT, this would go against the purpose of conducting a MANOVA: to test
the best picture based on multiple DVs (see Grice & Iwasaki, 2007)
• If you do, remember Bonferroni correction in the level of α (i.e., 0.05 divided by the number of DVs) is required to control for inflated Type I error.

Question 18

what are the statistical assumptions of MANOVA?

Answer 18

1. Multivariate normality
2. Homogeneity of Variance-Covariance Matrices
3. Linearity (linear relationships between all DVs, all covariates, and all DV-CV pairs)
4. Homogeneity of Regression (if using MANCOVA)
5. Absence of multicollinearity or singularity (This means using identical or very similar DVs. The current SPSS algorithm seems to ignore this problem, but singularity should stop the program; it does not make sense to use the same DV multiple times!)

Question 19

what measures are used in multivariate tests?

Answer 19

1. Wilks’ lambda [ λ ]: the variance in the combined DVs (discriminant functions) that is not accounted for by IV. (1 – λ) is the variance explained (i.e., η2).
   * The most commonly reported test statistic, but not always the best choice.
   * Gives an exact F-statistic.
2. Hotelling’s trace criterion [ T2 ]= usually reported for two-group comparisons.
3. Pillai-Bartlett criterion = pooled effect variances
   * Often considered the most robust and powerful test statistic.
   * Gives the most conservative F-statistic.
   Usually, Lambda, T2, and Pillai provide identical results (where there are only two groups).
4. Roy’s Largest Root
   * Gives an upper-bound of the F-statistic (only the first and the best discriminant function is used to calculate the variance explained).
   * Should be disregarded if none of the other test statistics are significant.

Question 20

what is repeated (mixed) measures of ANOVA?

Answer 20

• Involves at least one between-subject IV and one within-subject IV.
• The use of a within-subject factor provides a strong test of a treatment effect because variances due to individual differences can be first removed from the error term.
• For this reason, researchers interested in effects of treatment over time and those interested in how people react to a range of different stimuli tend to choose a within-subjects design.
• When effects of treatments over time (below a) or those of contrasted stimuli (below b) are expected to differ between groups, a mixed-design ANOVA is a suitable design.
• A hypothesis may predict that a series of measures will show a certain pattern (i.e., increase) in one group (e.g., experimental group) but another pattern (e.g., no change) in another group (control group).

Question 21

what are the assumptions od a repeated measures anova?

Answer 21

1. Normality
2. Homogeneity of Variance
3. Linearity
4. Sphericity (Compound Symmetry): this is relevant only where the within- subject factor involves more than two levels, as sphericity is the covariance between a pair of measures being the same for all pairs of measures.

(from PSY2PRM Week 11.2) If three covariances are basically the same, then sphericity is satisfied.

Question 22

what happens after violating sphericity?

Answer 22

1. 1. Use the MANOVA procedure provided in the ANOVA output.
      * Note that the multivariate procedure on offer is not as powerful as the ANOVA approach. You would wish to stick to the ANOVA approach as much as possible. To use the MANOVA option also requires that Box’s test (multivariate homoscedasticity) to be non-significant.
2. Make adjustment to the degrees of freedom in the F-test. SPSS provides three methods. JAMOVI the first two:
   * Greenhouse-Geisser – very conservative
   * Huynh-Feldt correction– less conservative than G-G.
   * Lower-bound – the lowest possible theoretical value for the data