Power and blocking Flashcards
When is unequal N too extreme?
Ratio of largest to smallest cell size > 3:1
In what ways do highly unequal N make analyses more unstable?
They increase both type 1 and 2 error.
def. power
Probability of correctly rejecting a false null hypothesis.
What are four interdependent factors affecting power?
- Significance level
- Sample size
- Mean difference
- Error variance
If you want to study small effects it will be highly likely you need ___ ?
Large N / sample size
What is the best way to determine power?
A priori
When can post hoc power analyses be useful? (2)
- In light of non-significance, to show a significant result would have required much larger N for the observed effect size.
- In light of non-significance, to show there was sufficient power to suggest a true case for the null.
A priori power estimates ask the question …
What N to achieve a given (.8) power?
Post hoc power estimates ask the question …
What power did I have given my N and effect size
What are four things we can improve to reduce error variance in a study?
- Operationalisation of variables (validity)
- Measurement of variables (internal reliability)
- Design of study - account for variance from other sources (blocking)
- Methods of analysis - control for variance from other sources (ANCOVA)
What are two applications of blocking?
- Reducing error variance, where the focal IV is underpowered.
- Detecting confounds, in the presence of a block-factor x IV interaction.
Would you expect a main effect of a blocking factor?
yes. This is a sign of good control variable, I.e. related to DV, but not IV.
What does it mean if a IV treatment effect does not generalise across levels of a blocking factor?
There is an interaction between IV and blocking factor, therefore a confound.
What is one advantage and disadvantage of a block x IV interaction?
Pro: the interaction soaks up more systematic variance, making the focal main effect more significant.
Con: this is outweighed by the focal IV effects DEPENDING on the moderator/confound (I.e. blocking factor)
When is there a loss of power due to including a blocking variable?
When there is a low correlation with block factor and DV, r < .2. (because of fewer error degrees of freedom)
