sample size and power analysis

Question 1

when do you call your treatment effective/ a success?

Answer 1

• determine that BEFOREHAND
  1. determine primary outcome
  2. determine what the difference should be = effect size (continuous measure: Cohen’s d/ binary measure: e.g., 25% difference)

Question 2

How do you know that effect to expect?

Answer 2

• active vs. non active: higher difference
• active vs. active: lower difference
  –> outcome in control group is also relevant: its about difference!
  1. educated guess: look in literature: what choice is likely (similar therapy in other patient group; other therapy in your patient group)
  2. if you really have no idea: choose what you think is clinical relevant
  ALWAYS CHOOSE LOWEST POSSIBLE OUTCOME (if you find a range; because if you demonstrate the lowest effect size, you automatically demonstrate the highest effect size)

Question 3

you know what effect size you want to demonstrate…

Answer 3

1. you can calculate how many people you need to include
2. to demonstrate the result with statistical significance
   =power analysis

Question 4

numbers you need to define in power analysis

Answer 4

1. effect size
2. alpha (type-1 error, false positive result), usually 5%
3. beta (type-2 error, false negative result), usually 20%
   - -> Power: 1-beta=(probability of detecting a difference when there is a true difference), usually 80%

Question 5

superiority trials, null hypothesis

Answer 5

there is no difference in the 2 groups

Question 6

inferiority trial question

Answer 6

• you want the scores in the 2 groups to be similar: how different can they be before you cannot call them similar anymore? = non-inferiority margin

Question 7

non-inferiority margin

Answer 7

e.g., d = +2 and d= -2 to say they are still equivalent (does the CI cross the margin?)

Question 8

superiority trial, H1

Answer 8

the intervention is delta better than the control condition

delta: difference between the two groups (often Cohens d)

Question 9

non-inferiority H0

Answer 9

the treatment is significantly worse than the control condition

Question 10

non-inferiority H1

Answer 10

the treatment is not more than delta worse than the control condition

Question 11

equivalence H0

Answer 11

the treatments are unacceptably different

Question 12

equivalence H1

Answer 12

the treatment is not more than delta worse and not more than delta better

Question 13

determine effect size and margins for equivalence and non-inferiority

Answer 13

non-inferiority: what difference would you accept between the treatments?
1. clinical relevance/ expert opinion
or 2. 95-95 method: previous studies (meta-analyses)
- take the lower boundary of the CI around the mean (M1)
- take the fraction of M1 to decide on the margin (e.g. 50%)
–> small margins: huge sample size!