L9 - One-Way ANOVA Assumptions and Designs, Error Rate Control

Question 1

What do the Brown-Forsyth and Welch tests do?

Answer 1

Brown-Forsyth and Welch tests are used when homogenity of variances is not met in a one-way between-subjects ANOVA.

They adjust the observed F value and degrees of freedom to make the actual false rejection rate closer to the nominal rate set by alpha.

Question 2

What are three statistical assumptions for one-way between subjects ANOVA

Answer 2

• homogeneity of variances
• independent observation
• balanced design

Question 3

What should be done if there is a discrepancy between inferences from Brown-Forsyth and Welch tests?

Answer 3

If there is a discrepancy between inferences using F* and W, when sample size differs AND variances are heterogenous, we should not reject the omnibus null hypothesis, if it is being used.

Question 4

What happens if there is a violation of homogeneity of variances, but sample sizes are the same?

Answer 4

The homogeneity of group variances assumption operates in a very similar way to that in the independent two-group design – if the sample size of each group is the same, or are almost the same, then inferences for both the omnibus test and planned comparisons are robust to mild-to-moderate violation of this assumption.

Question 5

If there is evidence that there is a violation of homogeneity of variance, what will the outcome be for the one-way design if it is a) balanced and b) unbalanced?

Answer 5

➢ Balanced: The Fcontrast/tcontrast tests for planned comparisons will generally still be robust when using MSwithin.
➢ Unbalanced: The use of MSwithin in testing planned comparisons will not be robust. –> need to use ROBUST TEST STATISTICS, using separate variances for the standard errors in planned comparison.

Question 6

What is an important assumption for within-subjects ANOVA using repeated measures?

Answer 6

the INTERVAL between TWO ADJACENT MEASUREMENTS must be the same for all people

Question 7

What is the SS within subjects further decomposed into?

Answer 7

SS within subjects = SS between occasions + SS individual x occasion interaction

occasion = variable/time point measurements

Question 8

What is the df for SS individual x occasion interaction?

Answer 8

(n-1)(k-1)

Question 9

What is the df for SS occasions?

Answer 9

k-1

Question 10

What is the df for SS within ?

Answer 10

N-n

Question 11

What is the df for SS between?

Answer 11

n-1

Question 12

How do we calculate the observed F statistic in within subjects ANOVA?

Answer 12

F = MS occ / MS error

Question 13

What are the two options for testing a null hypothesis using within-subjects ANOVA?

Answer 13

➢ The multivariate test – which does not assume sphericity of the covariance matrix among the dependent measures (this will not be approached in this subject); or
➢ The univariate test – which does assume sphericity.

Question 14

What kind of within subjects ANOVA does not assume sphericity?

Answer 14

Multivariate test

Question 15

What is sphericity?

Answer 15

Recall that the two important features in the dependent samples t test were:
➢ The use of difference scores between the set of paired measurements of each participant; and
➢ The strength of the correlation between the two sets of scores for reducing the size of the standard error.

The assumption of sphericity requires at a population level that:
➢ The variance of all difference scores between any two levels are the same; and
➢ The covariance between all sets of differences scores to also be the same.

A sufficient (but not necessary) condition for sphericity to hold is that the covariance matrix of the observed scores on all levels of the factor to show compound symmetry.

Question 16

What is compound symmetry?

Answer 16

A covariance matrix demonstrates compound symmetry if all variances along the diagonal of the matrix are the same value and if all off-diagonal covariances are also the same value (which implies that all correlations among the variables are equal).

Question 17

What is Mauchly’s test of sphericity?

Answer 17

➢ The null hypothesis for Mauchly’s test is that the covariance matrix demonstrates sphericity, therefore we do not want evidence to reject this null hypothesis (i.e. a small obtained p value is undesirable).
➢ The test itself can be affected by mild departures from normality in the DV scores, and therefore a significant result can reflect this occurrence rather than a departure of sphericity.

Question 18

What is epsilon?

Answer 18

Epsilon is a sample statistic measuring the degree of departure from sphericity.

If sphericity is present, population epsilon equals 1.
Its lowest possible value is 1/(k – 1); i.e. the lower-bound value in the third estimate provided – it can be ignored.

The Greenhouse-Geiser estimate is a more conservative estimate of the population epsilon that controls ER≤α.

The Huynh-Feldt estimate is more liberal.
If the Huynh-Feldt equals 1, then the obtained p value for Mauchly’s test will be >.05

Note: If in doubt, use Greenhouse-Geisser, as it is the middle estimate.

Question 19

Why is it important to run epsilon test?

Answer 19

The relevance of these estimates of epsilon is that violation of the assumption of sphericity makes the obtained F test no longer robust for the omnibus null hypothesis in within-subjects ANOVA and for planned comparisons—its becomes very liberal in these circumstances.

The epsilon estimates provide a way to adjust the degrees of freedom for the F test so that the false reject error rate is maintained close to alpha when the omnibus null hypothesis is true.

Question 20

How do we correct the degrees of freedom using epsilon?

Answer 20

By multiplying both degrees of freedom for the omnibus f test by the relevant epsilon estimate.

Question 21

What are orthogonal polynomial planned comparisons?

Answer 21

These are a wet of planned contrast weights used in one way between subjects ANOVA, to see what kind of change might be occurring across time.

is it linear? quadratic? cubic? quartic? etc.

can have k-2 planned orthogonal comparison weights.

–>set of orthogonal polynomial contrast weights completely account for all possible change b/c theyre ORTHOG.

–> prepackaged by SPSS, don’t need to work them out.

Question 22

How can we visually identify what ‘order’ polynomials may fit the data well?

Answer 22

by counting the amounts of ‘bends’ in the graph.

3 bends –> cubic

Question 23

What are the two different kinds of standardised mean contrasts for within-subject designs?

Answer 23

• Standardised group differences
• Standardised individual change

XECI will calculate point and interval estimates for group difference, but only point estimate for individ. change.

USE BONNETS DELTA or STANDARDISED D for individ change, and hedges’ g for group diff!

Question 24

What’s the difference between alpha value and false rejection error rate?

Answer 24

• ALPHA is set BEFORE undertaking the null hyp test

- FALSE REJECTION ERROR RATE occurs AFTER deciding to reject the null hyp

Question 25

What is alpha in notational form?

Answer 25

Pr ( Rejecting H0 | H0 is true!!!!)

Question 26

What is the Per comparison alpha value?

Answer 26

The probability of rejecting a null hypothesis, given that it is true, for each planned comparison.

This sets an upper bound to making false rejection errors of either kind OVER THE LONG RUN when using 2 or more planned comparisons.

Question 27

What is the family-wise false rejection error rate?

Answer 27

False rejection error rate amongst j planned comparisons over the long run.

ER fw = 1 - ( 1- ERpc)^j

Question 28

Under what circumstances is ERfw expected to be larger than ERpc?

Answer 28

ERfw is much higher than ERpc when many null hypothesis tests are undertaken

when 100 tests are being undertaken, the probability is almost 1.., when the null is true

Question 29

Why is using the explicit calculation for ER fw undesirable?

Answer 29

The comparisons must be independent, but in practice, comparisons are not independent.

Question 30

When are comparison’s not independent?

Answer 30

when. .
- weights for the two contrasts are NON ORTHOGONAL
- the SAME MSWithin value is being used to calculate their respective standard errors..

same MS within value is often used in practice, due to the homogeneity of variances assumption.

Question 31

How should ERfw be calculated when two or more comparisons are not independent?

Answer 31

1-(1-alpha pc)^j

Question 32

What is the bonferroni correction method?

Answer 32

A method for defining an upper bound on the family wise false rejection error rate when undertaking multiple planned comparisons.

Implies that the ER fw will be no larger than the sum of all the per comparison error rates.

alpha pc = 0.05/number of planned comparisons

Question 33

What are the implications of violating the assumption of sphericity for planned comparisons in a
within-subjects ANOVA?

Answer 33

The implications of violating the assumption of sphericity for planned comparisons in a withinsubjects
ANOVA is that the statistical testing associated with the planned comparisons are likely
to be false rejection rates higher than the nominal levels set by the defined α for each
comparison. This will mean that some contrast tests will be rejected more often when the null
hypothesis is true than what the rate is defined to be by the α value set for each comparison.