power analysis Flashcards
the significance criteria
alpha (risk of making a type 1 error)
usually set at .05
type 1 (alpha) error
when a difference or relationship is accepted when there isn’t one
type 2 (beta) error
when conclude there is no difference or relationship when there is one
effect size
aims to estimate the magnitude of an effect /population parameter (i.e. a difference or a relationship) independent of sample size
limitations of statistical significance testing
statistical probability cannot be used as a measure of magnitude because the significance level may be due to sample size or the effect size
cohens d guidelines for d (small, medium and large)
.20
.50
.80
cohen’s r guidelines (small, medium and large)
.10
.30
.50
cohen’s f2 guidlines (Small medium and large)
.02
.15
.35
power calculated based on values from
0-1
where 0.1= 10% chance of finding an effect
statisical power
the statistical power of a test is the probability of avoiding making a type 2 error (1-B)
recommended at .80
so the chance of making a type 2 error is 20%
ratio of type 1: type 2 errors
1:4
statical power depends on
effect size- larger more power
sample size- larger more power
precision of measures- reduce standard errors
why is the statistical power greater for a 1 tailed test
because more power is given to detect an effect in one direction by not testing the effect in the other direction
why is the statistical power greater for within subjects design
because the error variance ‘noise’ is reduced when using the same participants in both conditions of an experiment
effect size (in terms of power)
large effects are easier to detect and also less likely to make a type 2 error
if there is a big difference don’t need many people to detect it
sample size (in terms of power)
larger sample size gives a more accurate estimate of the means - smaller error bars (SD/ square root of n)
so easier to see a difference between two conditions
in the t test- how sample size affects power
error bars represent variability of the mean (i.e. the standard error of the mean)
larger samples result in smaller standard errors
therefore larger sample size the greater the power (due to smaller standard errors) so effects are easier to detect.
precision of measure (power)
more reliable measures provide more precise estimates of the latent variable therefore present less error variance/noise
and also smaller standard errors
better estimate of means=less error variance and therefore better chance of finding a true effect if there exists one
prospective power analysis called
‘a priori’
prospective power analysis
set sig criterion .05
set stat power .80
estimate your effect size as being small medium or large -cohens table (either based on previous research e.g a meta analysis, or from a pilot study)
the power analysis tells you how many ppt you need
retrospective ‘sensivity’
set sig criterion .05
set stat power .80
sample size is known
estimate minimum effect size that could have been calculated
(interesting to know why got non sig result-
given this design with this many ppt what could have been detected)
retrospective ‘post hoc’
calculating the statistical power - i.e. the likelihood of making a type 2 error
with the sig at .05
and sample size of x
what is the power of the test
is it underpowered?
retrospective ‘a priori’
set sig criterion .05
set stat power .80
estimate the required sample size for the effect size
e.g. if got nn significant result- can see how many more ppts would need
a priori
to do with sample size
sensitivity
what effect size could have been obtained
post hoc
testing power type 2 error
