Week Three - Power, Multiple Comparisons & ANOVA Assumptions Flashcards
How do we test assumption of Normality?
Can check by doing a plot and assessing if its normally distributed.
Shapiro-Wilk (want p value more than .05)
QQ plots (hoping for a straight line)
How do we test for Homogeneity of Variance?
Levene’s test (p value greater than 0.05)
What to do if you are uncertain about assumptions?
Can run different ANOVA (non-parametric) - if showing same sig then its fine, if not should say not 100% on results
What is power?
The probability of correctly rejecting the null hypothesis (probability of finding a difference between the means)
What are the bounds of a type I error rate set by?
The prescribed significance level chosed
When can a type II error occur?
When we have set the significance too strictly e.g., 0.001
When outliers exert an effect, increasing error variance
What does the error rate of a type II error depend on?
The population values of means and the within-group variance
Equation for power in written terms
The probability of finding a real difference = 1 - the probability of not finding the difference
Two ways to estimate power?
A priori power and post-hoc power estimate
What is a priori power estimate?
Is an estimate before the experiment is run, it helps to ensure we have adequate numbers of participants to detect any real differences between treatments
What is a post-hoc power estimate?
Estimation after the experimental data has been gathered, it estimates the likelihood of being able to replicate results if an experiment were repeated and it adds validity to a study
What do we use to obtain power estimate?
G power
How to obtain a priori power estimate?
Can use effect size from a previous similar study but not usually available.
Mathematical equations that determine the sample size needed to produce certain effect size. (can do through G power)
What is a critic of Bonferroni adjustment?
Making a BA may be too conservative and might make type II error
What can we do if we think BA is too conservative? What does this do
linear step up
it gives a critical p value to determine significance - can use og p value or t-test stat (bigger the better)
How do we combat type I error rates in contrasts for power?
Make bon adjustments
What do priori comparisons do?
Creates a smaller number of comparisons so type I error probability is reduced - usually do when we have a very specific hypoth
What can happen when we do multiple comparisons?
Can make type I error
When you are testing multiple post-hoc comparisons, what post-hoc test would be best?
Tukey or Holm (better at protecting against t2 errors)
If you have unequal sample size of homo of variance is violated, what post-hoc test should you use?
Games-Howell
When should we use Scheffe’s test?
When examining multiple, comples and post-hoc (unplanned) comparisons
