Stat models: ANOVA and non-parametric

Question 1

What is ANOVA

Answer 1

• ANOVA = analysis of variance.
• Investigates differences in means.
• In this course, we will look at one-way between group ANOVAs.
• Each participant in a different group.
• > 2 groups (if 2 groups, use a t-test).
• One-way refers to having one IV/factor.
  Each IV/factor will have >2 levels.
• The key ANOVA estimate is the F.
• F = (model variance) / (error variance).
• The model variance and error variance terms are calculated through “sums of squares”
• Remember, error variance often referred to as residuals.
• The larger the F, the more variance you are explaining in your DV by
  your IV, as compared to error
• The larger the value of F the more evidence we have against the null hypothesis.

Instead of averaging the squared deviations, which is what we do when calculating the variance, we just add them up.

When we talk about analysing variances in the context of ANOVA, what we’re really doing is working with the total sums of squares rather than the actual variance.

Question 2

Assumptions of ANOVA

Answer 2

Homogeneity of variance
Normality
Independence

These are for the residuals or error

Question 3

Effect sizes for ANOVA

Answer 3

• F is not an effect size measure.
• Partial eta2 or η2.
• η2 = 0.01 indicates a small effect
• η2 = 0.06 indicates a medium effect
• η2 = 0.14 indicates a large effect

Question 4

What a statistically significant F means

Answer 4

• This relates to a main effect of an IV/factor with >2 levels.
• This only means that there is at least 1 statistically significant
  difference between your groups.
• It does not tell you WHERE that significant difference(s) is(are).

Question 5

Post-hoc tests

Answer 5

• Tell you where the significant group difference(s) is(are).
• You can only run post-doc tests if you F is statically significant (p<.05 if
  you have a standard alpha of 0.05).
• Essentially you could achieve a similar outcome by skipping over the
  ANOVA and running six t-tests. But if you did this, you’d run into
  issues around multiple comparisons related to Type I error rate.
• Remember, if alpha = 0.05, then the Type I error rate = 5%.
• Remember, Type I error rate is the probability of rejecting the null hypothesis
  given its true; false positive.
• Central principle behind null hypothesis testing is that we control our
  Type I error rate.
• When we consider families of tests, e.g. the six tests required to test
  differences between our four groups (e.g. 6*0.05=0.30), then the Type I error rate inflates.

Question 6

Correction for multiple comparisons

Answer 6

• Adjusting the alpha to ensure you don’t inflate your Type I error rate.
• Bonferroni method, alpha/number of tests you’ll run.
• E.g. 0.05/6 = 0.008. Therefore alpha now 0.008 and you’ll only judge
  statistical significance (p value) when <.008.

Question 7

ANOVA Reporting

Answer 7

F(degrees of freedom group, participants) = esimate, p=x, η2 =x

Question 8

Why would we use a non-parametric test?

Answer 8

• If we violate any assumptions for parametric tests, e.g. our data is not
  normal.
• Most researchers aim to run parametric tests, as they generally have
  more statistical power, but sometimes we need to use non-parametric tests.
• Non-parametric tests have none or very few assumptions.

Question 9

Spearman correlation

Answer 9

• Spearman correlation coefficient is referred to as rho (ρ) or as rs.
• Works on ranks of data (unlike Pearson correlation).
• Can use same effect size cut-offs as Pearson correlation.
• Assesses strength of monotonic relationship between two variables,
  linear or non-linear.
• Assumptions that apply to Spearman correlation:
• At least one variable needs to be continuous.
• The other variable can be continuous or dichotomous.
• The relationship between the two variables is monotonic.

Question 10

What does monotonic mean?

Answer 10

If you order pairs of data, they constantly increase or consistently decrease.

Question 11

Wilcoxon tests

Answer 11

• Like the t-test, the Wilcoxon test comes in two forms, one-sample and two-sample.
• Can handle any type of data, where you want compare two groups.
• Mann-Whitney U Wilcoxon test for between-subjects.
• One sample Wilcoxon test for within-subjects.

Question 12

When do we run a Wilcoxon test?

Answer 12

When we want to compare two groups and out DV is non-parametric/not normally distributed. It’s a non-parametric version of a t-test.

Question 13

Levens test

Answer 13

Test the homogeneity of variance assumption has been violated. If statistically significant, perform a Welch one-way test

Question 14

A good experimental research design does what?

Answer 14

Minimises plausible alternative explanations

Question 15

Monotonic relationship

Answer 15

Cpnsistently increases or decreases, does not have to be linear.