Effect Size and Power Flashcards
What do effect sizes measure?
They indicate the proportion of the variance explained
What effect size measure do we use for T-Tests?
Cohen’s d
What effect size measure do we use for ANOVAs?
Eta and Partial Eta Squared
What is Cohen’s d for ANOVA?
The difference between the largest and the smallest group means scaled by standard deviation
What assumptions are made about the standard deviations when calculating effect sizes?
That it is constant across groups
What is Eta Squared and when do we use it?
- One-Way ANOVAs
- Same as R Squared
- Proportion of variance explained by your experiment
What is Partial Eta Squared and when do you use it?
- Factorial ANOVAs
- Proportion of variance that is uniquely explained by each IV
What is a power analysis?
Shows the power of a statistical test by checking its:
- Ability to detect an effect when it is actually there
- Ability to correctly reject the null hypothesis
What does the power of a test depend on?
- Sample size
- Effect size
- Criteria for significant
What are the dangers of underpowered studies?
- Lack of power to detect effect
- More errors likely
- Estimating powers across many studies is worryingly low
- Low power explains failure to replicate
Explain power as a function of sample size.
As sample size increases, so does power.
- More participants means increased chance of finding a significant effect if there is one
What number is considered good for power?
0.8
Explain power as a function of effect size.
Smaller effect sizes need more participants to achieve higher power.
If we know two of those things we can estimate the other.
What 4 ways can we use to estimate effect sizes?
- Guess
- Do a pilot study
- Find previous studies
- Find or conduct a meta-analysis
When would you guess an effect size?
When there is not much literature on your phenomenon
- Could use Cohen’s heuristics
- Not likely to be informative
When would you do a pilot study to estimate effect size?
- Run a pilot with fewer PPs to estimate effect size
- Doesn’t matter if the results are significant
- Can use this estimate to project how many PPs needed
How do you use previous studies to estimate effect size?
- Use their results to work out an expected effect size for your experiment
- Not exact but better than guessing
How do you use a meta-analysis to estimate effect sizes?
- If there are many studies you can calculate an average effect size across all the results
- Common in drug trials
What problems are there with power analyses?
- Garbage in - garbage out technique
- For really complicated designs (e.g. factorial) it is unlikely you will have sufficient precision in estimates of effect size
What is the garbage in - garbage out technique?
If you make up the numbers you enter, what you get out is not meaningful
What do you need to calculate how many PPs are required for a study?
- Effect size
- Power
- Alpha
What do you need to calculate minimum requires effect size detectable?
- Power
- Sample size
- Alpha
What do you need to calculate observed power?
- Sample size
- Effect size
- Alpha
