Statistics, study design and power Flashcards
What are the 6 key points of experimental design?
- To investigate a research question, apply scientific method
- Formulate a clear question
- Formulate a clear H0 and H1
- Plan the work required to test the hypothesis
- Collect, process and analyze the data
- Interpret the outcome in the context of the original question
What is a manipulative experiment?
- One or more factors that are being deliberately altered
- explore cause and effect
- example - Assess DNA recovery from samples stored in wet and dry environments
What is an observational experiment?
- Investigate links between variables of interest occurring in natural conditions
- Allows comparison between natural situations
- Example - what are the consequences of smoking on lung function?
What are the take home messages of statistical power?
- Don’t let your study be a waste of time
- Use the right number of samples
- Dont waste resources
- Too many samples are better than too few
- Allow for attrition (loss of data)
Why is it important to look at the right amount of samples?
If we accept the null hypothesis, we need to know that we looked at enough subjects to perceive accuracy
How often do we see type I error?
We see type I error 5% of the time - this is often not reproduced in replication studies
If the null hypothesis is false, what might happen if you reduce the type I error rate (alpha)?
We may increase the type II error rate (beta)
In a power calculation, knowing any three of these values allows calculation of the other…
- Effect size (ES)
- Sample Size (N)
- Probability alpha (P)
Most Power calculations relate to the relationship between ES and N
What does Effect Size measure
The strength of the result, it is solely magnitude based, it does not depend on sample size
Why does Power = 1-beta
Beta is the probability of incorrectly retaining a false null hypothesis
Power is the probability of incorrectly rejecting a false null hypothesis
What is P-value
relates to the likelihood that what you found is not due to chance
What are the 4 types of power analysis?
- A-priori …. N is calculated
- Post-hoc ……. Power is calculated
- Sensitivity …. ES is calculated
- Criterion ….. alpha is calculated
When is A - Priori completed and why is is needed
- Done in the planning stages to determine N
- Necessary to justify project resources and funding - minimize use of animals or risk to patients
When is Post-hoc completed and why is it needed?
Done on completion of the study to determine Power.
- Necessary to check that your expected and measured ES align.
- Did you have significant N to detect differences reliably?
What is a negative to Post-hoc
it is not always a good idea to complete this analysis because a non-significant result does not tell us whether the null hypothesis is true or false. To calculate power after the fact is to make an assumption that the null is false and that is not supported by data.
