RCTs and Power Flashcards
Why don’t we always increase power by changing significance level?
This will increase the probability of making a type I error
Type I vs Type II Error
Ways to increase power
- Increase n (narrow sampling distributions)
- Increase alpha
- Less variability in sample (out of your control)
- True parameter far from null hypothesis (out of your control)
Per protocol analysis
includes subjects who actually completed all aspects of the study as assigned
This analytical approach may overestimate benefit (resulting in bias), since it does not account for the reasons that study subjects stopped taking active therapy or began taking it if in the placebo arm. If these reasons for switching arms also predict outcomes, they are now confounders because they are associated with both exposure and outcome.
Intention to treat (ITT) analysis
includes all patients assigned to each arm at randomization (not just those who completed the trial, had good adherence, did not have side effects, etc.)
A better reflection of the likely benefit to a patient in the real world (outside the study) - and is the preferred primary analysis. Because exposure was randomly assigned, there cannot be confounding in this case
Ideally, we want the per protocol and ITT populations to be . . .
Ideally, we want the per protocol and ITT populations to be very similar
Individual randomization
each person randomly assigned, independently of the prior person’s assignment
Cluster randomization
groups of people within a pre-determined category are assigned together (e.g., all patients receiving care at a given hospital, one physician’s practice, children all attending the same school). This approach reduces the chance of contamination, but also reduces power, which means you need a larger sample size than randomization at the individual level.
Stratified randomization
investigators randomize subjects within strata of a third variable or risk group
Matching
Investigators match individual participants (or in the case of cluster randomization, similar hospitals or physician practices) based on certain observable characteristics (to create pairs of similar participants) and then randomize one of each pair to the treatment and one to the control group
Block randomization
Similar to stratified, but typically grouped into blocks according to a variable not of interest to the researchers (most often, time – e.g. block the first 100 patients recruited then randomize within that group so that 50 are assigned to intervention and 50 to control… then do the same thing with the next 100, etc. This keeps you from having the bad luck that, for example, most of your intervention subjects are assigned early and most of your control subjects assigned later in the study, if things like season, or secular changes in care would matter for your outcome.
RCTs are the strongest design in terms of . . .
. . . internal validity.
Efficacy vs effectiveness
Efficacy: An intervention’s effect under tightly controlled conditions
Effectiveness: Describes how the treatment works in actual practice.
Potential limitations of RCTs
- Generalizability
- Cost
- Feasibility
- Ethical issues
- Differences in groups despite randomization
Number needed to treat (NNT)
Calculation that estimates how many individuals need to be exposed to an intervention to see the outcome of interest
If the outcome of interest is adverse, e.g. a side effect of a medication, we might call it the number needed to harm (NNH). If the exposure is a screening test, we might call it the Number Needed to Screen (NNS).
Sample size power formula
More detailed Type I error vs Type II error diagram
