3A. Statistical power, effect size and sample size Learning Outcomes

Question 1

what is statistical power?

Answer 1

the ability to detect an effect if it is present

e.g. a power of 80% means we’d get a statistically significant difference 80% of the time and 20% of the time we wouldn’t even if there is a real difference

Question 2

what 4 things affect statistical power?

Answer 2

• sample size
• critical significance level
• effect size
• type of stats test

Question 3

what type of tests have greater stats power, parametric tests or non-parametric tests?

Answer 3

Parametric tests

Question 4

what is the critical significance level?

Answer 4

probability of falsely rejecting the null hypothesis when it is actually true

Question 5

what is effect size

Answer 5

can mean the treatment effect as such the difference/ratio between two measurements

e.g. task completion time was measured for 2 groups A and B, the difference between the means was 1.5s which is the effect size

Question 6

what is the equation/definition for standardised effect size?

Answer 6

standardised effect size = mean difference / pooled standard deviation of pooled observations