Statistical Power Flashcards
What is the Baseball analogy?
If a baseball player knows is well-trained in skills, but does not now the rules of the game and how to interact with his teammates, he still does not win games. In statistical power, the same thing happens; books are written on only the power computations, but don’t take the context in consideration.
Which characteristics are crucial in hypothesis testing?
Hypotheses must be designed a priori, null-hypotheses must be set (= denying the theory), and most important: they must be very specific.
What entails equipose in regard to hypotheses?
Equipose means that you need enough background information to support your hypotheses, but not too much to disregard the “need” for your research into these hypotheses (e.g., the researchers make it look like they are already convinced that their hypotheses are true; you need to be objective in the interpretation of your results). There needs to be room for the hypotheses to be false.
How do you decide on which statistical tests to use?
You base this on the evaluation of the performance of each proposed test (H0 is true = significance level) (Ha is true = power)
What is a type-I-error?
The type-I-error is equal to the significance level (often 0.05), which means the chance that you reject your null hypothesis (aka there is no difference), while this is in fact true. This is also called the “false-positive” error.
Why do we use effect-sizes?
The rejection of H0 (so, there is an effect/diference), does not say anything about the clinical significance/implications of this effect. So, the ES is used to give a representation of clinical/scientifical importance.
What is a type-II-error?
The type-II-error is equal to the power of a significance test (1 - α), which means the chance that you reject the alternative hypothesis (there is no effect), while there actually is an effect. This is called the false-negative error.
What is a type-II-error?
The type-II-error is equal to the power of a significance test (1 - α), which means the chance that you reject the alternative hypothesis (there is no effect), while there actually is an effect. This is called the false-negative error.
On which characteristic does the power of an α-test depend?
On the effect size in the population
What are the 3 forms of ES? And which one is the most important for clinical practice?
- The true population ES (which is unknown)
- The sample ES (also known al SRD, estimate of the true ES)
- The critical ES (SRD*) = the range of population effect sizes that would be considered of clinical/practical importance, based on background information.
What entails an adequately powered α level significance-test?
An α-level significance test is adequately powered, if the probability of a significant results is higher than the prespecified value for any ES greater than the critical ES.
What do you do when there are questions about the validity of t-tests?
Then you shift to non-parametric tests, since these do not rely on the assumptions of t-tests (normality, equal variances etc.); therefore, you protect the validity with only limited power-loss.
What is the problem with statistically significant results?
Only significance levels say nothing about the clinical significance of an effect, so by just finding that your H0 can be rejected, you still don’t know anything about how large the effect is. For this, you need to use the effect size.
What is the problem with non-significant results?
Non-significant results are often interpret as “failed” studies, while finding that H0 is true is also very important. Most often however, nonsignificant results indicate a poor study design/poor measure.
What is “Cherry-picking”?
Cherry-picking is the phenomenon where researchers replace their original outcome measure with one that has been found statistically significant, during the research.
