16. Power Analysis Flashcards
What is power?
The probability that you will find an effect when an effect is present
Why is power important?
Prevent time wasting
Can answer the questions you’re interested in
What are the different factors effecting power and how does it effect it?
Sample Size
- Power increases as sample size increases
Effect Size (The detectability of the alternative hypothesis as it compares the distance between the alternative and null hypothesis to the variability in the data - magnitude of data needs to be considered)
- Power increases as the parameter values moves further into H1 values away from H0
Significance level (alpha)
- Power increases as the significance level increases
What is the problem of increasing the significance level of alpha?
It leads to more Type I errors - a higher chance of incorrectly rejecting true null hypothesis (false discoveries)
What is required to come to a conclusion about effect size?
Typically requires some experience with the measure involved and knowledge of past research and to figure out what significant results were reported
What is type I error?
Reject the null hypothesis when null = true
What is type II error?
Failing to reject when null is false
What does a represent in terms of power (not alpha)?
Tolerance for type I error as it reflects probability of its occurrence
In hypothesis testing, which symbol/connotation resembles power?
1-B (as represents ability to detect effect if it exists)
How is power related to sample size?
Power relates to what sample size is big enough for a study
Depends on the study design, sample characteristics, expected effect size, desired level of precision, costs and benefits of different results, significance level
What is power by simulation and why is it needed?
For complex designs, a simulation can be used to estimate power
If enough simulations are conducted, simulation-based estimates of power will approximate estimates of power from analytical solutions.
Why is it needed?
Sometimes the analytic solutions are not accurate
Sometimes there is no analytic power solution
What is analytical power analysis?
Take known numbers (alpha, power,n, and effect size)
Solve for unknown number
Via.
Calculate power or calculate sample size
Can something be referred to as underpowered?
No, not technically correct
What is post-hoc power?
Calculating power after collecting data
Use effect size from collected data
What are the typical conventions of analytical power analysis?
Alpha fixed at 0.5
Common conventional power is .8 (80%) chance of detecting h1 if it it exists so type II error - .2
Effect size based on previous studies or there are conventional set amounts depending on what test of power you are carrying out
