Power Analysis Flashcards
A statistical tests power is the probability that it will…
Avoid Type 2 error/correctly pick up on an effect
This hypothesis states that the difference between the two variables is zero
Null
One motivation for conducting a study is to provide evidence that the hypothesis is true. We do this by showing…
The null hypothesis is unlikely
A Type 1 error is where
False positive
A Type 2 error is a
False negative
“Rejection of the null, when it is in fact true”
Type 1 error
What is an effect?
What was found
What is effect size
Measures strength of result - magnitude-based
Difference between EFFECT SIZE and SIGNIFICANCE?
Effect size does not depend on sample size
The p value indicates if what was found is g..
Generalisable to a population
Effect size does not depend on
Sample size
If we find a non-significant p-value despite a large effect, this is likely because
Small sample
A prospective power analysis shows us that a certain ________ are needed to yield a significant result
Number of pps
Sufficient power to find statistical significance minimises
Chance findings
A power analysis is where one of several _______________ can be calculated given others
Statistical parameters
There are ___ parameters involved in a power analysis
4
4 parameters involved in a power analysis
(i) Alpha
(ii) Power
(iii) N
(iv) Effect size
Coefficient alpha is the p…
Probability of finding significance where there is none - Making a Type 1 Error
Coefficient alpha is usually set to
.05
Power is usually set to
.80
Power is the probability of finding
True significance
Power is the probability of avoiding
A type 2 error
Power analysis
The parameter usually solved for is…
N
Expected effect size may be ascertained from… (4)
- Pilot study
- Findings from similar study
- Field defined ‘meaningful effect’
- Educated guess
Alpha is usually set to ___ and power is usually set to ___
.05
.80
When trying to ascertain effect size, if there are no similar studies…
Start with pilot study
There are FOUR TYPES OF POWER ANALYSIS
Priori
Post-hoc
Criterion
Sensitivity
For a priori, you’d compute ___. The rest is given
Sample size (N)
For a post-hoc, you’d compute ____. The rest is given
Power
For a criterion, you’d compute ___. The rest is given
Alpha
Which type of power analysis is USUALLY DONE
Priori
A 72% power indicates that you have a 72% chance of
Avoiding making a Type 2 error/Picking up on an effect
Power is increased when a researcher increases
Sample size/Effect Levels/Significance Levels
_______ is increased when a researcher increases sample size/effect levels/significance levels
Power
How to calculate effect size
M1 + M2 / SD (collective)
A power analysis is where
One of several statistical parameters can be calculated given others
Reporting a prospective priori order
- Pps
- Effect size
- Alpha
- Power
Reporting a sensitivity order
- Pps
- Effect size
- Alpha
- Power
Reporting a post-hoc
- Effect size
- Pps
- Alpha
- Only achieved 73% power
How do you begin reporting a post-hoc
The effect size for this study was calculated as
How do you begin reporting a retrospective priori
The effect size for this study was calculated as
Statistical power depends on (3)
- Effect size
- Sample size
- Precision of measures
Why does statistical power depend on precision of measures?
More reliable measures = smaller SE’s/more precise variable estimates
Effect size
The ______ the better
Larger
Sample size
Larger sample size =
Smaller SE’s
Error bars
Small samples will have
Large error bars
Error bars
Easier to estimate when
Error bars are smaller
Large error bars = _____ sample size
Small
Prospective power analysis
We don’t want to recruit too many pps
As this could result in significant result regardless of effect size
Other than a Priori, you could calculate required sample size
Using Cohen’s table
Why does sample size affect power
Larger = smaller SE’s
Small, non-overlapping error bars
