Power & Blocking Designs Flashcards
What is a a Type 1 error?
Incorrectly rejecting the null hypothesis (false positive)
Finding a significant difference in the sample that doesn’t exist in the population
What is a Type 2 error?
Incorrectly retaining the null hypothesis (false negative)
Finding no significant difference in the sample when one exists in the population
What is power?
Correctly rejecting the null hypothesis
The degree to which we can detect treatment effects when they exist in the population
What is the probability of making a Type 1 error?
5%
α = .05
What is the probability of correctly retaining the null hypothesis?
95%
1 - α
How is power calculated?
1 - β
What is the probability of making a Type 2 error?
β
What are 4 ways to increase power?
- Increase sample size
- Increase alpha (but increases Type 1 error rate)
- Find a larger effect
- Reduce error term
What are 4 ways to increase power?
- Increase sample size
- Increase alpha level (but increases Type 1 error)
- Focus on larger effects
- Decrease error variance
Why would you care about power before running a study?
To make sure there is enough power to detect predicted effects
Why would you care about power after running a study?
To increase power to detect a difference in a population that is there but was not observed in the sample
What are estimates needed to calculate power a priori?
- Estimate of effect size
- Estimate of error (MSerror)
Typically using previous research
What values are needed to calculate power post-hoc?
- Effect size
- Error (MSerror)
- N in the study
From the dataset
What is effect size?
It measures the overlap of distributions in terms of standard deviations and is closely related to power
What are ways to decrease MSerror?
- Improve operationalisation of variables (increase validity)
- Improve measurement of variables (increase reliability)
- Improve design of study by accounting for variance from other sources (blocking)
- Improve methods of analysis by controlling for variance from other sources (ANCOVA)
What do blocking designs do?
They introduce a control variable to account for additional variance in the DV. It increases power by reducing error variance
How do you set up a blocking design?
- Divide participants into groups to make a continuous variable ‘categorical’ (low, med, high)
- Randomly assign participants within each block to levels of the IV
What is stratified random assignment?
Randomly assigning participants within each level of the blocking factor to different levels of the IV
What is a sign that the blocking variable is a good control variable?
If there is a main effect of the blocking variable
What is a sign that the blocking variable is a confound?
Is there is an interaction between the blocking variable and the IV
What are some advantages of blocking?
- Greater power
- Can check interactions for potential confounds
- Equates treatment groups better than a completely randomised design
What are some disadvantages of blocking?
- Practical costs
- Loss of power if poorly correlated with the DV
- Grouping can cause loss of information
