7. F-Distribution - Assumptions of ANOVA

Question 1

What does the Null hypothesis tell us?

Answer 1

the proposition that all treatment means are equal

i.e. the IV has no effect on the DV

Question 2

what odes the alternative (research) hypothesis tell us?

Answer 2

the proposition that at least one treatment mean differs from another treatment mean

i.e. the IV does have an effect on the DV

Question 3

what is a sampling error

Answer 3

when populations have different means even through the null hypothesis is true

Question 4

what should F equal when null hypothesis is true?

Answer 4

1

however this may not always be true due to sampling error

Question 5

what does simulation study involve?

Answer 5

repeatedly drawing a same-sized set of number of random samples of the same n from a single population then calculating F for each test (this is called the Monte Carlo siulation)

Question 6

what is the alpha level?

Answer 6

the probability level we adopt for concluding that it is unlikely that sampling error caused the observed difference in means

Question 7

what is the conventional alpha level

Answer 7

a=5% (.05)

Question 8

what do we conclude if we find that F_observed is greater than our F_critical?

Answer 8

then we conclude that there is a significant difference between at least two of the treatment means this reject the null hypothesis

Question 9

what is the process of hypothesis testing based on?

Answer 9

probability not certainty.

and will therefore sometimes make errors

Question 10

how often will we incorrectly reject the null hypothesis?

Answer 10

a% of the time

Question 11

what is a type 1 error

Answer 11

when we incorrectly reject the null hypothesis when the null is true

Question 12

what is a type II error rate

Answer 12

the probability of getting an F value less than the F critical when null is false.

Question 13

how are type II errors signified?

Answer 13

β - beta

Question 14

what happens when we lower the type I error rate?

Answer 14

we suffer an increase in the type II error rate (and vice versa)

Question 15

what are the three assumptions of Individual groups ANOVAs?

Answer 15

1. independence of observations
2. normality of distributions
3. homogeneity of variance

Question 16

what happens when we break assumptions of ANOVAs?

Answer 16

will lead to either an inflated or deflated estimate of the true BG or WG variability. thus inflate or deflate the obtained F-value (F=BG/WG)

Question 17

what is does the independence assumption state?

Answer 17

that it is not possible to predict one score in the data from any other score.

Question 18

how can this assumption be adequately met in a between groups experimental design?

Answer 18

• random assignment of participants to groups (level of IV0
• random selection of participants from population of interests
• Each participant contributes to only 1 score in the analysis (may be the mean of many observations)
• each participant’s score is independent - i.e. not influenced by any other participant’s score

Question 19

what does the normality assumption state

Answer 19

the normality assumption states that the samples are drawn from normally distributed populations and that the error component is normally distributed within each treatment group (level of IV)

Question 20

What is robust to breaches of the normality assumption?

Answer 20

ANOVAs

Question 21

what are the conditions for an ANOVA to be robust to normality assumptions?

Answer 21

• there are similar number of participants in each condition
• there are at least 10-12 participants in each condition
• the departure from normality (skewness or kurtosis) is similar in each condition

Question 22

how to we determine whether the assumption of normality is breached?

Answer 22

can either:

• inspect frequency histograms for each experimental condition
• skewness
• kurtosis
• use SPSS

Question 23

what do measurements of skewness indicate about normality of a distribution?

Answer 23

0 = normally distributed data (or any symmetrically distributed data)
positive/ negative values = distribution negatively or positively skewed

Question 24

what do measures of kurtosis indicate about normality of a distribution?

Answer 24

3 = normally distributed data (or any distribution that dont have more outliers than normal distributions)

Question 25

what is a less common method of fixing problems with normality?

Answer 25

transformation of the data

Question 26

what is an outlier

Answer 26

an extreme score at one or both ends of our distribution

Question 27

what can outliers do you our ANOVA>

Answer 27

can influence our results as our anova is based on a ration between and within group variance. Our variance measure is based on our SD from the mean, so an outlier can inflate our measure of variance, and it can also impact on our mean

Question 28

what are the solutions to problems of outliers

Answer 28

• remove from the data
• transform the data to remove the influence of other outliers
• use a non parametric test
• bootstrapping techniques
• run analysis with and without outliers and see if they affect results. If not report this and report ANOVA as usual (use with caution)

Question 29

what is a way to transform the data to remove an outlier?

Answer 29

using the winsorized samples technique

Question 30

what does the winsorized samples tecnhique do?

Answer 30

samples replace extreme scores with the most extreme score left in the tail of the distribution

e.g.

original data:

3, 7, 12, 15 … 32, 33, 50, 75

Windsorised

3, 7, 12, 15 … 32, 33, 33, 33

Question 31

what is the homogeneity of variance assumption

Answer 31

As MS_within is a pooled error term, we need to ensure that variance within each of treatment conditions / groups is similar.

Question 32

what is the rule of thumb for variance sizes?

Answer 32

the largest variance should be no more than 4 times the smallest variance

Question 33

what are breaches of homogeneity of variance assumptions composed of?

Answer 33

very unequal cell sizes

Question 34

what can breaches of homogeneity of variance assumptions impact?

Answer 34

type 1 error rate

Question 35

what is the test for breaches of homogeneity of variance on SPSS?

Answer 35

Levene’s Test

Question 36

how can we deal with breaches of homogeneity of variance?

Answer 36

1. If you have equal cell sizes and breach is minor (i.e. largest group variance less than 4 × smallest), you can run an ANOVA as it is robust to minor breaches of homogeneity
2. Run the ANOVA but use a lower alpha level to control for the possible impact on the type I error rate
3. Use an alternate statistical test which does not have the homogeneity assumption (e.g. nonparametric test)
4. Transform the data to remove the heterogeneity and run the ANOVA on the transformed data
5. New computer intensive methods such as bootstrapping (not covered in this unit).

Question 37

what does lowering the alpha level do?

Answer 37

reduces the type 1 error rate. thus the effect of the breach of homogeneity of variance can be reduced

Question 38

what are parametric tests?

Answer 38

tests like t and F which make assumptions about the distribution of scores

Question 39

what are nonparametric or distribution free tests?

Answer 39

tests that have less restrictive assumptions about the distributions used

Question 40

how do most nonparametric tests work?

Answer 40

by converting each score to a rank thus removing the assumption associated with inferential statistics

the ranks are spread out evenly so the shape of the distribution will always be rectangular (=no normality assumption and no problems with outliers)

there are specific rank-order tests for various hypothesis-testing situations

in rank order tests we are coparing mean ranks rather than mean scores to test the hypothesis. that samples are drawn from identical populations. Hence using medians rather than means is more appropriate when describing group differences

Question 41

what statistic does the Kruskal-Wallis test use?

Answer 41

chi square

Question 42

what does a significant chi square indicate?

Answer 42

significant difference between groups (because it is smaller than the significnace number)

Question 43

what does data transformation involve?

Answer 43

performing an identical mathematical operation on all the scores.

Question 44

what do the transformations do to the distribution of scores

Answer 44

it changes the shape of the distribution of scores

Question 45

what can a suitable transformation do>

Answer 45

reduce heterogeneity of variance

achieve normality

Question 46

what are some types of common transfomraitons?

Answer 46

• Logarithmic (strong skews, outliers and breaches of homgenity)
• Square-root (positive skew)
• Reciprocal or reflect (negative skew )
• Trimmed samples (outliers or heavy tailed kurtosis)

Question 47

what do logarithmic transformations do to the size of values?

Answer 47

Log transformation compress large values but have less effect on small values.

Question 48

what will logarithmic transformation do to the spread of scores

Answer 48

it will reduce the spread of scores this reducing skewness and decreasing the variability int he samples with large SDs

Question 49

what are the steps in doing a transformation?

Answer 49

1. Identify problem (heterogeneity or skewness).
2. Find the transformation which minimises this problem. Check the assumption again on the transformed data.
3. Do not look for the transform which maximises F, but the one that minimises the assumption breach
4. Perform ANOVA using these transformed scores as the DV.
5. Transformed data is harder to interpret because it is no longer expressed in the original units of measurement.
6. Only do a transform when absolutely necessary.
7. You can run ANOVA on transformed and original data, and if you get the same result, report the original data as they are easier to interpret.

Question 50

what are the advantages of data transformations?

Answer 50

can use parametric techniques on transformed scores

Question 51

what are the disadvantages

Answer 51

• May not successfully normalize scores.
• Can distort meaning of data and result in loss of information.
• Risk of Type I and Type II errors (and hence power) is unclear.

Question 52

what are the advantages of non parametric tests?

Answer 52

• Can use regardless of the shape of the original distributions.
• No parameter estimations so no assumptions.

Question 53

what are the disadvantages of non parametric tests>

Answer 53

• Cannot be used for many complex situations.
• Logic of ranks does not always work when there are many “tied” scores.
• Can distort meaning of data and result in loss of information.
• Risk of Type I and Type II errors (and hence power) is unclear.