ANOVA Flashcards
What is ANOVA?
Statistical test to compare means of more than two groups
Multiple pairwise comparison is…
repeating multiple t-tests (this gives the same results as an ANOVA)
How many IV levels are there in an ANOVA?
at least 3
Notation for number of levels in an ANOVA
k
A pairwise comparison is…
One possible t-test out of the many that could be done
A pairwise comparison is done to…
determine which treatment affects the dependent variable
Formula for how many pairwise comparisons there will be
k(k-1)/2
Each pairwise comparison needs a pooled Standard deviation in the denominator, we call this the…
Mean Squared Error
What do we use for the Mean Squared Error for ANOVA?
All the sample standard deviations of the ANOVA (gives a more accurate p-value)
Formula for an individual pairwise comparison
The risk for a type I error increases for each per comparison you make…
we call this the per comparison (PC) error rate
- Type I error occurring somewhere in at least one of the comparisons is higher than the PC rate
The risk of committing a Type I error at least once in a whole family is called the…
Family wise( (FW) error rate
- This is substantially higher than the PC rate
- This is because each test you add to the family of tests you expose a fresh possibility of committing and error
Statiticians perfer the FW error rate, not PC rate to be set to…
Alpha=0.05
Two ways to control family wise error
1) Use smaller PC error rate in each test so FW doesn’t blow up as much
2) Limit the number of comparisons
Bonferroni formula is used for…
Familywise error rate
FW = PC x C
e.g.,
= 0.05 x 10 = 0.5
Bonferroni formula with PC error rate being smaller example
FW = 0.005 x 10 = 0.05
Using a more stringent error rate for the per comparisons makes the FW error rate 0.05 which is more reasonable
Bonferroni correction formula
Used to determine the PC rate we want
PC=FW/c
e.g., PC = 0.05/2 = 0.025
This new corrected amount is called the Bonferroni correction
When using the Bonferroni formula it controls for family wise, but it is achieve at the price of reduced _______
power
When you decrease the alpha on a statistical test you also decrease the ______
power
There is a balance between controlling for family wise and retaining _____
power
There is another procedure that uses a formula for estimating FW that is more accurate than Bonferroni. This is because Bonferroni….
overestimates the true FW
Sidak formula
A better estimate of FW is given this formula
FW=1-(1-PC)^c
e.g., FW=1-(1-PC)^c
FW=1-(1-0.5)^30=.785
Unlike the Bonferroni formula the Sidak formula never predicts error rates greater than 100%
Therefor for large number of comparisons you should use sidak
Sidak rearranged to find the proper PC error rate in order to find any FW error rate that we want
Sidak Correction
The benefit of this is that they don’t require the PC error rate to be reduced as much as Bonferroni. Retaining power
Sidak vs Bonferroni. How many comparions?
Bonferroni for 10 or less comparisons
Sidak for more than 10 comparisons
What do you have to do before making comparisons?
Choosing how many you are going to compare. This is done before looking at any data. If you look at the data you have to run ALL of the tests
P-hacking
Choosing to only run certain tests to get significant results. This leads to type I errors
Type I error
False positive
Type II error
False Negative
A priori coparisons
The comparisons that are planned to test before hand
Post-hoc comparisons
Comparisons that are not planned a head of time. Procedure involving post hot tests must perform all possible comparisons
A priori tests have…
Slightly inflated FW rate
It is still common to reduce FW error by reducing PW in some way
The way in which you choose to reduce PC to bring residual FW under control determines what a priori test you are performing
Post hoc tests…
Force you to perform all comparisons amoung means
There are still many methods of reducing PC in order to limit FW error rate.
The method you choose determines what post hoc procedure you are performing
You could use either Sidak or Bonferroni
A prior procedures
Systematic methods of planning comparisons for apriori tests
- Multiple comparisons against control group (k-1) pairwise comparisons (called Dunnett’s test)
- Contrasts which are generalization of pairwise comparisons
What are contrasts?
Compare 2 groups of means rather than just two individual means
Lumping together certain IVs then performing a t-test
What must the sum of contrasts coefficients add to?
0
calculating a contrast value
Calculating a contrast value example
If contrasts had no affect the value would equal what
0
If its not 0 a t-test will further explain whether it is an accident
T test formula for a contrast
The contrast df is quite large becasue it depends on the design using Spooled
N-k
Continue at Learning Spooled
