Sample size and Statistical Power

Question 1

Name the six study design principles.

Answer 1

• Well-defined RQ
• Clearly specified research hypotheses
• Clearly defined population
• Determine which measures to use for IV(s) and DV(s)
• Determine optimal experimental design
• Determine how many participants to recruit

Question 2

Type I Error

Answer 2

We reject the null hypothesis based on sample information when it is actually true in the population

Question 3

Type II error

Answer 3

We accept the null hypothesis based on information in our sample when it is actually false in the populaion.

Question 4

Power expressed in relation to type II error

Answer 4

Power = 1 - (type II error)

Question 5

Controlling errors

Answer 5

• We control type I error probability directly through choice of significance level
• We control type II error probability indirectly through sample size + other design factors
• All else held constant, increasing sample size will increase power
• Increasing power leads to reduced probability of making a type II error

Question 6

Pooled SD /

Answer 6

• Average variance between individuals within groups
• Can reduce pooled SD by selecting measures with greater between subject homogeneity

Question 7

Power normally depends on four factors

Answer 7

• α level or statistical significance – larger α → more power
• magnitude of the effect of interest in the population (Cohen’s effect size) – larger effect size → more power
• the level of variance of the population – smaller SD → more power

•the sample size – larger samples → more power
Cohen’s

Question 8

Power function for the unpaired T-test

Answer 8

• propoertions that are allocated to each group. Note power will be optimised by allocating an equal number of participants to each group
• z-scores that correspond to each group
• MCID (when you don’t have appropriate you can consider % changes)
• pooled SD

Question 9

Power afterthoughts

Answer 9

• While we express the unstandardized effect size as μe - μc it is only the size of the difference that matters, not the values of the individual means
• The biggest problem in prospective sample size calculation is usually finding an estimate of the SD for a scale (DV) that you have not used before

Question 10

Power function for the proportions test

Answer 10

• Has the same elements as the function for comparing two means but expressed in proportion/binomial terms
• Assumes equal samples sizes but can accomodate inequal samples
• Uses the unstandardised effect sizes
• Unlike when comparing means, for proportions it is not just the difference in proportions but what the individual proportions are matters as well.