Week 6 - Assumptions of ANOVA

Question 1

What is a type 1 error?

Answer 1

A type one error is falsely concluding that you’ve found something significant when in fact there was nothing going
on.

Question 2

How do you know what the type one error rate is

Answer 2

The alpha that I said, that criterion in which I decide that something is significant is identical to my type one error rate.

if alpha is 0.05 my type one error rate it 5%

Question 3

What is a type 2 error?

Answer 3

When you falsely accept the null hypothesis and reject the true research hypothesis and thereby conclude there’s nothing going on, whereas in fact there is something going on.

Type II error rate is signified by beta (β)

Question 4

If you change your alpha level to lower what happens?

Answer 4

Decreases chance of getting a type 1 error but increases chance of getting a type 2 error

Question 5

What are the three ANOVA assumptions?

Answer 5

1. DV should be measured on a metric scale
2. Independence of observations
3. Normality of Distributions
4. Homogeneity of variance

Question 6

What does the assumption stating DV should be measured on a metric scale mean?

Answer 6

If your DV is not measured on a metric scale (if its not a true number) don’t use an ANOVA

metric scale = has to behave like a true scale –> have to be able to add, subtract and divide

Question 7

What is the ‘independence assumption’

Answer 7

• States that it is not possible to predict one score in the data from any other score

-the only way you can know the probability of events happening is if you know they are independent of one another

• A requirement for calculating any p-value

Question 8

In a between groups design, how do you assure the independence assumption is met?

Answer 8

• Random assignment of participants to groups (levels of IV).
• Random selection of participants from the population/s of interest (particularly important with some types of IV where random allocation is impossible).
• Each participant contributes only 1 score to the analysis (this may be the mean of many observations).
• Each participant’s score is independent – i.e. not influenced by any other participant’s score

Question 9

What is the normality assumption

Answer 9

States that
- the samples are drawn from normally distributed populations and

• the error component is normally distributed within each treatment group (level of IV)

Question 10

How can you check the normality assumption? (what are the assumptions you have to meet to consider data to meet the normality assumptions)

Answer 10

• See if there are a similar number of participants in each condition.
• See if there are at least 10-12 participants in each condition.

-Check the departure from normality (skewness or kurtosis) is similar in each condition (don’t want one condition massively positively skewed and another condition massively negatively skewed)

Question 11

To see weather the normality assumption has been breached what is the best thing is to inspect?

Answer 11

frequency histograms for each experimental condition

Question 12

For a completly normal population, what would your skewness value be?

Answer 12

0: normally distributed data (or any symmetrically distributed data)

• Positive/negative values: distribution skewed positively/negatively

Question 13

For a completely normal population, what would your kurtosis value be?

Answer 13

0 (on ANOVA) normally distributed data (or any distributions that don’t have more outliers than normal distributions)

*In mathematical terms Kurtosis should equal 3 for a normal distribution, however SPSS subtracts the 3 to give 0.

Question 14

What are the two types of tests on ANOVA that check normality assumption

Answer 14

Shaprio-wilks
Kolmogrove Smirdoff

Question 15

What are outliers?

Answer 15

*Of more potential impact on our statistics than the shape of our distribution per se is the problem of outliers.

*An outlier is an extreme score at one or both ends of our distribution

Question 16

Why are outliers a problem?

Answer 16

▪ Can inflate our measures of variance

▪ Can also affect our mean (drags distribution towards the mean)

▪ Potentially caused by spurious (incorrect) data
– e.g., eyeblink on the part of the participant or an error in the procedure

Question 17

In an ANOVA do all your sample sizes need to be the same?

Answer 17

No this is a myth

could have 50 in one group and 150 in another if you wanted

Question 18

How do you deal with outliers?

Answer 18

• First decide why it is an outlier… it may reflect an out-of-range value or a participant who is not part of targeted populations

Some solutions to problems of outliers:
* Remove them from data (common, but potential problematic solution)

• Transform data to remove the influence of outliers (make the outliers smaller)
• Use a non-parametric test (e.g., based on ranked data)
• Run analysis with and without outliers and see if they affect your results. If not report this and report ANOVA as usual

Question 19

What is Homogeneity of Variance

Answer 19

-The assumption stating that all comparison groups have the same variance

• You need to check that the variances within your distributions are approximately equal so that it is safe to pool them
• A rule of thumb is that the largest variance should be no more than 4 times the smallest variance
• Breaches of homogeneity can affect the type I error rate
• Breaches of the homogeneity assumption are compounded by very unequal group sizes.
• There are a number of tests for breaches of homogeneity - e.g. Levene’s Test provided in SPSS

Question 20

What does a a significant Levene’s test (p <.05) mean?

Answer 20

means variance in each group is significantly different and the homogeneity assumption is breached

*you want it to be non-significant so the assumption is met

Question 21

How do you deal with breaches of the homogeneity assumption?

Answer 21

1. If you have equal group sizes and breach is minor (i.e., largest group variance isn’t more than 4 × smallest), you can run an ANOVA as it is robust to minor breaches of homogeneity
2. Run the ANOVA but use a lower alpha level to control for the possible impact on the type I error rate
3. Use an alternate statistical test which does not have the homogeneity assumption (e.g., nonparametric test)
4. Transform the data to remove the heterogeneity and run the ANOVA on thetransformed data
5. Perform a robust test (e.g., Welch test or Brown-Forsythe test)

Question 22

How does lowering the alpha level help deal with breaches in homogeneity?

Answer 22

➢ Breaches of homogeneity in ANOVA cause overestimation of the true value of F.

➢ This means we make more type I errors.

➢ Lowering the alpha level (e.g., .025 rather than .05) reduces the type I error rate.

➢ Thus, the effect of the breach of homogeneity can be reduced by using a lower alpha level.

Question 23

What is a distribution Free test?

Answer 23

If you ever are stuck with one of these problems of normality or a problem of homogeneity of variants, look around for a distribution free alternative (also called nonparametric tests or distribution free tests)

These tests have less restrictive assumptions about the distributions used

Question 24

How do distribution Free test/ nonparametric tests work)

Answer 24

• Most of these work by converting each score to a rank.
• Ranks are spread out evenly so the shape of the distribution will always be rectangular (= no normality assumption and no problem with outliers).
• There are specific rank-order tests for various hypothesis-testing situations.
• In rank order tests we are comparing ranks rather than scores.
• Some also compare medians rather means for describing group differences.

*converts a number (say a score) to a rank. So if you are ranked 2nd highest your number would be 2. Therefore they don’t have the ability to be large or small to drag the mean in one way or another

Question 25

What is the non-parametric alternative to independent samples ANOVA

Answer 25

Kruskal-Wallis H test

Question 26

What is data transformations?

Answer 26

Data transformations involve performing a mathematical operation on all the scores on your DV

• These transformations change the shape of the distribution of scores.

A suitable transformation can
➢ Reduce heterogeneity of variance.
➢ Achieve normality

**not changing the nature of the data just changing the scale.

Question 27

What are some types of common transformations

Answer 27

➢ Square-root (positive skew)
➢ Logarithm (strong skew)
➢ Reciprocal (extreme skew)
➢ Trimmed samples (outliers)

Question 28

What are the steps in doing a data transform

Answer 28

1. Identify problem (heterogeneity or skewness).
2. Find the transformation which minimises this problem. Check the assumption again on the transformed data.
3. Do not look for the transform which maximises F, but the one that minimises the assumption breach
4. Perform ANOVA using these transformed scores as the DV.
5. You can run ANOVA on transformed and original data, and if you get the same result, report the original data as they are easier to interpret (check weather if you reject or accept the null in both)

Question 29

Normality and Homogeneity Summary

Answer 29

➢ Independence assumption is a design issue, but normality and homogeneity assumptions cannot be totally controlled by good design.

➢ So, when performing an ANOVA, always check normality and homogeneity.

➢ If normality breached: Ask whether the breach tells you something important about the data (eg. bi-modality may mean that the IV has different effects for different people, or
outliers may indicate sampling issues).

➢ ANOVA is robust against breaches of normality provided group sizes are reasonably equal, n in each group is not too small, and skewness is in similar direction in each group. Under these conditions we can run ANOVA

➢ If we have small or uneven group sizes we can transform data and run ANOVA with transformed data or run non-parametric test

Question 30

How is the independence assumption different to the normality and homogeneity assumption?

Answer 30

Independence assumption is a design issue, but normality and homogeneity assumptions cannot be totally controlled by good design