Week Three - Power, Multiple Comparisons & ANOVA Assumptions

Question 1

How do we test assumption of Normality?

Answer 1

Can check by doing a plot and assessing if its normally distributed.

Shapiro-Wilk (want p value more than .05)

QQ plots (hoping for a straight line)

Question 2

How do we test for Homogeneity of Variance?

Answer 2

Levene’s test (p value greater than 0.05)

Question 3

What to do if you are uncertain about assumptions?

Answer 3

Can run different ANOVA (non-parametric) - if showing same sig then its fine, if not should say not 100% on results

Question 4

What is power?

Answer 4

The probability of correctly rejecting the null hypothesis (probability of finding a difference between the means)

Question 5

What are the bounds of a type I error rate set by?

Answer 5

The prescribed significance level chosed

Question 6

When can a type II error occur?

Answer 6

When we have set the significance too strictly e.g., 0.001

When outliers exert an effect, increasing error variance

Question 7

What does the error rate of a type II error depend on?

Answer 7

The population values of means and the within-group variance

Question 8

Equation for power in written terms

Answer 8

The probability of finding a real difference = 1 - the probability of not finding the difference

Question 9

Two ways to estimate power?

Answer 9

A priori power and post-hoc power estimate

Question 10

What is a priori power estimate?

Answer 10

Is an estimate before the experiment is run, it helps to ensure we have adequate numbers of participants to detect any real differences between treatments

Question 11

What is a post-hoc power estimate?

Answer 11

Estimation after the experimental data has been gathered, it estimates the likelihood of being able to replicate results if an experiment were repeated and it adds validity to a study

Question 12

What do we use to obtain power estimate?

Answer 12

G power

Question 13

How to obtain a priori power estimate?

Answer 13

Can use effect size from a previous similar study but not usually available.

Mathematical equations that determine the sample size needed to produce certain effect size. (can do through G power)

Question 14

What is a critic of Bonferroni adjustment?

Answer 14

Making a BA may be too conservative and might make type II error

Question 15

What can we do if we think BA is too conservative? What does this do

Answer 15

linear step up

it gives a critical p value to determine significance - can use og p value or t-test stat (bigger the better)

Question 16

How do we combat type I error rates in contrasts for power?

Answer 16

Make bon adjustments

Question 17

What do priori comparisons do?

Answer 17

Creates a smaller number of comparisons so type I error probability is reduced - usually do when we have a very specific hypoth

Question 18

What can happen when we do multiple comparisons?

Answer 18

Can make type I error

Question 19

When you are testing multiple post-hoc comparisons, what post-hoc test would be best?

Answer 19

Tukey or Holm (better at protecting against t2 errors)

Question 20

If you have unequal sample size of homo of variance is violated, what post-hoc test should you use?

Answer 20

Games-Howell

Question 21

When should we use Scheffe’s test?

Answer 21

When examining multiple, comples and post-hoc (unplanned) comparisons