Lecture 12 - RCT Critical Appraisal Flashcards
What are some advantages of RCTs?
- Rigorous evaluation of single variable (drug treatment vs placebo/active comparator) in a precisely defined patient population
- Prospective design - data collected on pre-specified outcomes after researcher decides to do study
- Limits bias by theoretically comparing two identical groups (if they are randomized, the two groups should be similar)
- Results lend themselves to meta-analysis (i.e. combining numerical results of several similar trials)
What are some disadvantages of RCTs?
- Expensive
- Time consuming
- Money source dictates research agenda
- Surrogate end-points used to limit cost and time required for study
- Failure of randomization and/or binding
- Unethical to randomize
What do we need to consider when asking if we can believe the results?
1) Blinding
- Patients, clinicians, assessors
2) Outcomes
- Soft and hard outcomes
- Surrogate outcomes
- Composite outcomes
- Primary and secondary outcomes
- Intention to treat
- Follow-up
3) Assessing Randomization
What is on the RCT Checklist for Are the results valid?
1) Was the study original?
2) What was the study about?
3) Was the study design reasonable?
4) Were patients randomized?
5) Were patients in treatment and control groups similar with respect to known prognostic factors?
6) Was randomization concealed?
7) Were patients analyzed in the groups to which they were randomized?
8) Was follow up complete?
Describe
1) Was the study original?
- Is the study bigger, longer or otherwise more substantial?
- Is the population studied in a different way (age, gender, ethnic groups)?
- Is the clinical issue addressed of sufficient importance and relevance?
Describe
2) What is the study about?
- Patients more or less sick than patients you see in your practise?
- Do they represent a different ethnic group or lifestyle than the patients you see?
- Is the care they receive similar to the care they might receive in your setting?
- Did they have “real life” comorbidities?
- Are they of similar age and gender to patients you might care for in your practise?
- Inclusion criteria
- Exclusion criteria
Describe
3) Was the study design reasonable?
- Was the intervention appropriate?
- Was the comparator appropriate?
- How was the outcome measured?
- soft vs hard outcome
- surrogate outcomes
- composite outcomes
- primary vs secondary outcomes
Compare soft vs hard outcomes
- Hard outcome is death (easy to tell if someone is dead or not)
- Soft outcome is something that is more subjective (ex. pain)
Describe
4) Were patients randomized?
- Observational studies - larger treatment effects
- Limitation to our understanding of prognostic effects
- Selection bias - treatment assignment
Power of randomization:
- Balance of known and unknown factors
- Does it always achieve equal prognostic groups?
- Large versus small sample size
- Compromises to randomization
Describe
5) Was randomization concealed?
- Appendectomy - open versus laproscopic
- Chemotherapy
- Pharmacy preparation of blinded medication
- Unacceptable randomization - sequential allocation, clinic visits by day
Types of randomization:
- Simple randomization
- Randomized permuted blocks
- Stratified randomization
Remote randomization - call to remote randomization centre
Were patients aware of group allocation?
- placebo effect
- color, taste, consistency
- unblinding the blind
- if there is a recognizable SE, need to find a way around it
Where clinicians aware of group allocation?
- Double blinding
- Difference in post-randomization care
- Ethics: CHF ACEi and BB ???
- Effective blinding eliminate possibility of potential care bias
Were outcome assessors aware of group allocation?
- Triple binding (patient, caregiver, and the people who are analyzing the results of trial are blinded)
- Soft vs hard outcomes (if my outcome is hard, then i’m not as concerned with blinding)
- Maybe only blinding possible
- Blinded adjudication committee
Describe
6) Were patients analyzed in the groups to which they randomized?
-Omit results from patients who do not take their assigned treatment?
-Related to prognosis - non-compliance
-Intention to treat analysis:
1-Treatment outcomes - analyze our results using the intention to treat outcomes or something
2-Safety outcomes - don’t do ITT on safety outcomes, we analyze safety outcome where we don’t include people that didn’t use the drug or used a smaller amount
Describe
7) Were patients in the treatment and control groups similar with respect to known prognostic factors
- Check if randomization has done its job
- With randomization, we are trying to achieve two groups that look the same. We will check Table 1 or an RCT, there will be a descriptive analysis of the 2 groups and you can look at the factors that would be important for the disease study and see if there is a difference between the 2 groups.
Do we need to look for statistically significant differences between RCTs?
NO - the % that it’s due to chance is 100% because we randomized it that way
Is the P value useful in the context of RCTs?
Nope
Describe
8) Was follow up complete?
- Status of all patients
- > lost to follow > compromise to study validity
- how much is too much?
- 20% threshold ?
WTF
