Power Analysis Flashcards
What is a Type I error?
When we accept a difference or relationship when in reality there isn’t one
What is a Type II error?
When we conclude that there is no difference or relationship when in reality there is one
What is the usual significance criterion?
α (the risk of making a Type I error) is usually set at .05
What are the limitations of statistical significance testing?
Statistical probability can’t be used as a measure of the magnitude of the result as this may reflect either the effect size or the sample size.
What is the effect size?
A measure of the magnitude of a result which is independent of sample size
What is statistical power?
The power of a test is defined as the probability of avoiding making a type II error
The recommended level of power is .80, the probability of making a type II error is .20
What is the ratio of chances of making a Type I : Type II error?
1:4 ratio
What does statistical power depend on?
Effect size
Sample size
Precision of measures
What do error bars represent?
The variability of the mean, they are the standard error of the mean
What does having a larger sample mean?
Smaller standard errors, which means the larger the sample, the greater the power (due to smaller standard errors)
SO effects are easier to detect
What do more reliable measures provide?
More precise estimates of the latent variable (less variance/noise)
Will also result in smaller standard errors
What is a prospective power analysis (“a priori”)?
- Set significance criterion (.05)
- Set statistical power (.80)
- Estimate effect size (on the basis of previous research, conduct pilot study, decide on small/med/large effect size)
- Calculate required sample size
What is a retrospective power analysis (“sensitivity”)?
- Set significance criterion (.05)
- Set statistical power (.80)
- Sample size is known
- Estimate minimum effect size that could be detected
What is a retrospective power analysis (“post hoc”)?
- Calculate effect size
- Significance criterion is known
- Sample size is known
- Calculate statistical power of test
What is a retrospective power analysis (“a priori”)?
- Calculate effect size
- Set significance criterion (.05)
- Set statistical power (.80)
- Estimate required sample size for the effect size
How do you use G*Power3 to conduct a power analysis?
Tests –> Test class –> Design
Choose one of the five types of power analysis
Provide input parameters
Click ‘calculate’ to obtain output
What must be included in the report of a power analysis?
All the inputs determining the results
- Effect size e.g. d or r or other
- Significance level (α)
- Required power
- Sample size
How do you report an a priori prospective power analysis?
An a priori power analysis indicated that it would be necessary to recruit __ participants per group in order to detect an effect size of d = [d value], with alpha set at .05, at 80% power.
How do you report a sensitivity retrospective power analysis?
A power analysis indicated that with a sample of __ in the experimental group and __ in the control group, it would be possible to detect a minimum effect size of d = __, with alpha set at .05, at 80% power.
How do you report a post hoc retrospective power analysis?
The effect size for the experiment was calculated as d = __. A power analysis indicated that with a sample of __ in the experimental group and __ in the control group, and with alpha set at .05, the analysis only achieved __% power.
How do you report an a priori retrospective power analysis?
The effect size for the experiment was calculated as d = __. A power analysis indicated that it would be necessary to recruit at least __ participants per group to detect this effect size, with alpha set at .05, at 80% power.
