drug lit - RCT :) Flashcards
Which one of the following best classifies a clinical trial?
A
Primary literature
B
Secondary literature
C
Tertiary literature
D
Quartile literature
A
Primary literature - correct :)
A RCT is the highest level of medical evidence.
A
True
B
False
False - because they fall at number 2 and meta-analysis is number 1
Background
- GOLD standard - study design to measure and quantify differences in effect of an intervention versus a control
- Required for all new drug entities prior to FDA approval (most are RCTs)
- Basis for clinical practice
– Compiled into treatment and clinical guidelines
– Results change clinical practice - Pharmacists routinely use randomized controlled trials (RCTs) to:
– answer clinical questions
– Stay up-to-date with pharmacy knowledge - Pharmacists must be able to:
- Find appropriate literature
- Evaluate literature
- Explain findings - RCT are primary literature
- Hierarchy of Medical Evidence
Hierarchy of Medical Evidence
- reviews, meta-analysis
- randomized control trial
- non-randomized controlled trials
- cohort study
- case-control study
- case report/case series
- background info., expert opinion
RCT in a snapshot
sample population
randomization
interventional group - outcome
comparator group - outcome
compare differences in outcome
statistical significance - is the difference due to chance
clinical significance - is the difference meaningful
Clinical significant can differ in opinion - is that difference meaningful
RCT components
Abstract
- Brief overview of the research, synopsis, 200-300 words
- Generally includes: introduction (justify why the clinical trial is being done, hypothesis to study), methods, results, conclusion
Introduction
- Disease/drug background information
- What is already known
- Establishes rationale for current study
- Includes clinical trial objective/hypothesis
Methods - we will spend most of our time here
- Study design
- Inclusion/exclusion criteria
- Intervention/control groups
- Randomization
Results
- Number of participants
- Participant characteristics
- Dropouts - if any participants dropped out
- Endpoints quantified
- Safety assessment
Discussion
- Author’s interpretation of results
- Comparison to other studies
- Strengths/limitations - what are the flaws and strengths of the study
Conclusion
- Final conclusion of the study
- No new information introduced
Acknowledgments
- Other contributors
- Funding sources
- Peer-review dates/manuscripts acceptance date
References
- Citations of information included from other sources
RCT Evaluation
Evaluate the study
- strengths vs. __limitations
- As limitations goes up, the strengths go down
Evaluate the therapy
- _benefits vs. _harms
- More harms, benefits goes down
Evaluating the ABSTRACT
- Give it a “quick” read, do not base entire decisions,
- is the rest of the article worth reading?
What is the intervention?
What is the comparator ?
What is the OUTCOME?
What is the population?
Evaluating the BACKGROUND
- Background/Introduction section
- Study rationale
- Previous data
- Trial objectives
Consider the journal the article is published in
- Peer-reviewed (Strength) - meaning the work is sent to experts and provide comments to the author and sends the comments to the author who makes changes and revises work
- Reviewed, prior to publication, by experts in the field
Reduces publication of:
Inappropriate methods/design
Poorly written
Does not meet needs of journal audience
Reputable (Strength)
- Affiliated with a professional organization
- Impact factor - the higher, the more times the article is cited
- # of times the journal was _cited in 2 years = impact factor / # of articles _PUBLISHED in 2 years
Example major medical journals
- New England Journal of Medicine (impact factor: 176.1)
- The Lancet (impact factor: 202.7)
- Journal of the American Medical Association (impact factor: 157.3)
Evaluating the METHODS
Internal vs. external validity
Internal Validity - inside the study
- The degree to which a study accurately measures the relationship between the variables it intends to investigate, while minimizing the influence of confounding variables
- Think inside the study: How well the study was conducted?
Internal validity is about ensuring that the way a trial is set up and carried out does not lead to errors, so we are confident that the results accurately reflect the relationships between the independent and dependent variables
- Impacted by:
Study design
External Validity - outside the study
- The extent to which the findings of a study can be applied to real-world situations or populations beyond the specific conditions of the research
- Think outside of the results: External validity involves considering how well the findings of a study can be relevant and useful in situations and with people different from those in the specific research, ensuring the broader relevance of the study’s conclusions
- Impacted by:
Enrollment procedures
Treatment protocol
Inclusion and exclusion criteria
Duration and follow-up
Treatment adherence
Concomitant treatments
Baseline characteristics
Study design
Variables (see Dr. Carey’s material for more information on variables)
- Any data point or characteristic that can be measured or counted
- Independent variables
changed or manipulated by the researcher
Examples: inclusion/exclusion criteria, intervention, control group - Dependent variables
Affected by the independent variable
Examples: trial outcomes
Confounding variables
extraneous or third variable variable that is related to a study’s independent and dependent variables
Unmeasured third variable the effects both the cause and the affect
Make sure to control for it
Prospective vs. retrospective
Prospective: study participants are followed over time and data is collected at follow-up visits
RCT is prospective (weakness)
Retrospective: data from participants is collected from the past
Often used in observational studies (strength)
Randomization
goal: achieve balance between the groups, control for
Separates RCT from other study designs (strength)
Participants have equal chance of being placed in either group
The goal of randomization is to achieve balance between the groups in a clinical trial
Allow investigators to control for confounding
A confounding variable is a type of extraneous variable that is related to a study’s independent and dependent variables
Types of Randomization
simple
blocked
stratfied
simple randomization
Each individual in a study has an equal and independent chance of being assigned to each group
Methods:
Flipping a coin
1:1 ratio
Random number tables
advantages
- Simple
- Unbiased allocation
disadvantages
- Predictable
- does not account for confounders
- confounders may not distribute evenly
blocked randomization
Participants are grouped into blocks, and within each block, randomly assigned to the treatment or control group
Block size can vary
Randomization sequence can be changed with each block
Pick a set of letters and as the participants come in, they are assigned according to the order of the set
Ex: ABAABA
First person is A, Second person is B etc
advantages:
Treatment groups will be equal
Tend to have uniform distribution of __confounders
disadvantages
- Complex
- Vulnerable to predictability if block sequence not changed
- Harder to know
- Eventually predictable
stratified randomization
Participants from a population are grouped into strata based on a clinical feature that can influence or confound the outcome
Participants are then randomized from each stratum into the intervention or control group
advantages
- Confounding variables are evenly distributed between the groups
disadvantages
Complex
Potential for reduced generalizability
Some strata may be small
Which one of the following is the weakest form of randomization?
A
Simple
B
Blocked
C
Stratified
D
They are all equally weak
A
Simple
Which forms of randomization evenly distribute Confounding variables? (Select ALL that Apply)
A
Simple
B
Blocked
C
Stratified
B
Blocked
C
Stratified
Multi-centered study design
Higher external validity because we will get more diverse patients
More than one site recruiting patents
Concern for internal validity
- What variables are left up to investigator discretion? - can lead to how patients are treated or followed
- Will it be consistent across different sites?
Single-centered study design
Higher internal validity
Concern for external validity
Will the population be diverse enough to extrapolate to a general population?
Only one site
Parallel RCTs
Participants stay in originally assigned treatment group for the duration of the study
Crossover RCTs
Participants start with one treatment then switch to the opposite treatment with a washout period in between
Allows participants to serve as their _own control, and so can use a smaller sample size
Blinding (also known as masking)
Is used to prevent bias in clinical trials
If participants are aware of treatment allocation in can bias:
- Expectations
- Adherence
- Treatment-seeking behavior
- Assessment of effectiveness
If investigators are aware of treatment allocation in can bias:
- Different treatment, attention or attitudes toward subjects
- Assessment of outcomes
If outcome assessors are aware of treatment allocation it can bias:
- Assessment of outcomes
Once bias is introduced into a study from one of these potential sources, there is no analytical technique that can reliably correct for this _limitation - we reduce the bias from ever being introduced
Levels of blinding - also known as masking used to prevent bias:
unblinded blinded
single blinded
double blinded
triple blinded
Unblinded (also known as open-label)
_participanets and researchers - so all parties__ aware of treatment allocation
advantages:
Extension phase of RCT
Early stage trials
Rare diseases
Ethical reasons
disadvantage:
_everyone is aware and may raise bias
Single-Blind
_participants unaware of treatment allocation but researchers are aware
advantages
Bias from participants reduced
disadvantages
Potential for unblinding - because the side effects may be evident
Challenges for recruitment
Double-Blind
_participants__ and _researchers unaware of treatment allocation
advanatge
Bias from participants and investigators reduced (more rigorous)
Improves internal validity
disadvanatge
Potential for unblinding - because the side effects may be evident
Resource intensive
Triple-Blind
___participants____ and researchers and data-assessors (outcome assessors) unaware of treatment allocation
advantage
Bias from participants and investigators and assessors reduced (most rigorous)
Improves internal validity
disadvantage
Potential for unblinding
Resource intensive
Which level of blinding involves keeping participants, investigators, and outcome assessors unaware of treatment allocation?
A
Single-blind
B
Double-blind
C
Triple-blind
D
Open-label
C
Triple-blind - correct :)
Enrollment - where do we get out patients from
Recruitment methods
Incentives
Target own patients - MGH may recruit their own patients
Advertisements
Other sources - physician referrals
Enrollment can impact internal and external validity
Internal validity
- Too few patients = underpowered
Selection bias
- Subjects that meet inclusion/exclusion criteria are not enrolled
Examples:
- Only enroll patients that will be adherent, may overestimate efficacy
- Only enroll patients they perceive will do well
External validity
Not representative of the target population
Underrepresentation of minority populations
Inclusion/exclusion criteria
____________: participant demographics that must be present in order for the person to be enrolled into the trial
Should match population being studied
Defined objective criteria
____________: characteristics that prevent a participant from enrollment into the trial
- Patients at risk or who have confounding variables
- Special populations (pregnancy, cancer, children, etc.)
- Risk of adverse events, contraindications to study drug
- Characteristics that alter pharmacokinetics (movement of the drug) (renal/hepatic function)
Important for extrapolation (generalization) of results to general population (_external validity___________)
Subjective inclusion/exclusion criteria lowers _our internal validity___________
Allocation concealment (Strength)
The person or system responsible for randomizing people to the intervention or control group:
- Does not know what the next treatment allocation will be and
- Conceals the results of the allocation from others
Allocation concealment is not the same as blinding
- Allocation concealment is preventing the knowledge of the randomization sequence
- Blinding is preventing the knowledge of treatment assignments once participants are randomized
Minimizes risk which increases internal validity
- investigator cannot influence group assignments
- Minimizes chance of blinding being revealed
Methods of allocation concealment
- Centralized randomization done with an automated computer system or off-site statistician
- Interactive Voice Response Systems (IVRS)
- Interactive Web Response Systems (IWRS)
- Centrally controlled
- Pharmacy controlled
- Sequentially numbered opaque sealed envelopes (SNOSE)
Not concealed if:
- Non-opaque, non-sealed envelopes
- Investigator controls allocation
- Any method where the investigator or participant is aware of the allocation scheme or pattern
Allocation concealment is a different concept to blinding. It means that the person randomising the patient does not know what the next treatment allocation
Note that allocation concealment is completely different from blinding. The allocation seeks to eliminate selection bias during the process of recruitment and randomization, whereas blinding seeks to reduce performance and ascertainment bias after randomization.
True or False: Allocation concealment is the same as blinding
A
True
B
False
False - correct
Which one of the following is NOT a means of allocation concealment?
A
SNOSE
B
Interactive voice response system
C
Investigator control
D
Pharmacy controlled
C
Investigator control - correct
Intervention and comparator
Intervention
- Therapy under investigation
comparator
Therapy used as a comparison to intervention
- Placebo
- Active control
- Another medication
- Medical intervention
Historical data
Double-dummy
Used when comparing different dosage forms or frequencies
Two different placebos used to maintain blinding
Example
Intervention: IV active drug, placebo oral medication
Comparator: IV placebo, active oral medication
Titration period
Study phase in which the medication is slowly increased (given) to:
Minimize side effects
Reach a certain affect
Blood pressure
INR
Etc.
Run-in phase (also known as a lead-in phase)
Phase completed by participants prior to their enrollment in the study
Usually brief (2-4 weeks)
Placebo run-in (also known as a washout period)
Washes out effects of prior treatments
Active run-in
Assess adherence
Adverse events
Participants that do not meet prespecified criteria
Need to determine if run-in phase leads to selection bias
Ancillary treatments
Treatment allowed outside of defined groups
Some participants may receive
Often needed for ethical reasons (_strength)
Based on “judgement” of physician (__weakness__________)
Leads to differences between groups (_weakness___________)
Confounding variable (_weakness___________)
Adjunctive treatments
Treatment given to both groups
Often needed for ethical reasons (_strength___________)
Not a confounding variable (_strength___________)
Trial duration
Duration needs to be long enough for outcome to appear
Too short = lowers internal validity
Too long = may lower external validity
Very rare
Planned vs. actual duration
Early termination
Usually due to ethical reasons
Consider impact on results
_weakness___________: termination results in inability to rely on results
_neutral/strength___________: despite early termination, had statistically significant results
Funding
Authors required to disclose source of funding and conflicts of interest
Sources of funding:
Pharmaceutical industry
University medical centers
Nonprofit foundations
Government agencies
Pharmaceutical industry funding
Necessary for new medications
Evaluate involvement in:
Study design
Data monitoring and analysis
Manuscript preparation
Monitoring
How/when participants were followed
Monitoring occurs at predetermined intervals
Data collection method determined before trial starts
Should be consistent/feasible in clinical practice
Relates to external validity___________
Should be the same for all participants
Relates to _internal validity___________
Objective monitoring parameters (_strength___________)
Measurable
Subjective monitoring parameters (_weakness___________)
Self-reported
Potential for ____________: a systematic error that occurs when participants do not remember previous events or experiences accurately or omit details
Adherence
Should match practice (80% = _strength___________)
Sometimes manipulated or controlled (_weakness___________)
Data and Safety Monitoring Board
Part of some larger trials
Members of board blinded to treatment allocation
primary Endpoints (what is this?)
Primary endpoint
Prespecified outcome that directly addresses the hypothesis of the study
The basis of trial design, including sample size, power analysis, etc.
Secondary endpoints
Examined if the primary outcome is statistically significant
Trial is _not powered___________ to secondary endpoints
Interpret with caution
Should not base clinical decision on this
Disease oriented vs. patient oriented
Disease oriented evidence (DOE)
Nonclinical evidence that addresses factors that do not have a direct impact on patients
E.g., cholesterol levels, HbA1c
Do not always correspond to the _morbidity or mortality___________ results that were presumed
Patient oriented evidence (POE)
Measures outcomes that patients care about
E.g., quality of life, quantity of life
Patient oriented evidence that matters (POEM)
Outcomes that patients and clinicians care about
E.g., cardiovascular events, vision loss, kidney failure, quality of life
types of endpoints
singular
coprimary
composite
surrogate
Singular
One primary endpoint
advantage:
Simple
Statistically efficient
Reduced sample size
disadvantage:
Incomplete assessment
Risk of misinterpretation
Coprimary
Two or more primary endpoints of equal importance
advantage:
Useful if demonstration of two or more outcomes is necessary to establish clinical benefit
More comprehensive
Address multiple objectives
disadvantage:
more complex to interpret
Increased risk of _type ii error (saying that there is no difference hewn it really does exist - false negative)__________
Increased sample size requirements
Composite
A combination of two or more individual endpoints into a _single outcome measure___________
Purpose: capture a broader array of clinical events, provide a comprehensive assessment of efficacy, or enhance statistical power
advantage:
Increased overall event rate
Efficient
disadvantage:
Complicated interpretation
Risk of misinterpretation
Effect often smallest for most important component and largest for less important components
Must look at individual components to see what is __________ statistical significance
Prone to post-hoc analyses
Surrogate
A substitute or proxy measurement
advanatge
Quick
Inexpensive
disadvantage
Risk of misinterpretation
May not predict clinical benefit
Must be validated
Which one of the following is an example of disease oriented evidence?
A
Mortality -
B
Quality of life
C
Fatal myocardial infarction
D
Blood pressure
E
Nonfatal stroke
D
Blood pressure - correct
Which one of the following endpoints is a combination of ≥ 2 individual outcomes combined into a single outcome measure?
A
Singular endpoint
B
Co-primary endpoint
C
Composite endpoint
D
Surrogate endpoint
C
Composite endpoint - correct :)
Endpoint adjudication
adjudication = verification
Important in trials with clinical outcomes
Reduces risk of bias that can occur when the interpretation of results is different with different medical providers
Third party blinded to treatment allocation that verifies that an outcome has occurred
Statistics
See Biostatistics class notes for information pertaining to:
Variables
Descriptive statistics
Alpha (α), type I error, p-values
Beta (β), type II error, power
Confidence intervals
Calculations: ARR, NNT/NNH, RRR/OR/HR, RRR/RRI
Types of analysis
intention-to-treat (ITT) Analysis
_all patients___________ randomized are included in statistical analysis
Accounts for loss to follow-up, death, lack of adherence, treatment failures, etc.
High external validity
Gold standard for RCT
Strength
Modified ITT Analysis
All patients randomized that received _at least 1 dose___________ of study medication are included in statistical analysis
High external validity
Strength
Per Protocol Analysis
Only patients that _completed treatment___________ are included in the statistical analysis
May overestimate efficacy and underestimate safety
Low external validity (weakness)
Helpful if used as a sensitivity analysis to verify ITT results (strength)
A priori (before) vs. post hoc (after)
Terms used to describe the timing and nature of decisions related to data analysis or hypotheses
A priori
Decisions and hypotheses that are established __before__________ the data is collected or analyzed
Strength
More robust
Less prone to bias
Helps control for type I errors (false positive)
E.g., primary and secondary outcomes, sample size calculations, statistical analysis plans
Post hoc
Decisions and analyses made _after___________ the data has been collected and without pre-established plans
Weakness
More susceptible to biases, as researchers may be influenced by the observed data
Overinterpretation or introduction of false associations
Inflated risk of type I errors
Hypothesis generating
Statistical tests
Continuous Data
number of groups: 1
_normally distributed___________
number of groups: 1
__one sample test or dependent/paired t test__________
number of groups: 2 (intervention and comparator)
_independent/unpaired student t-test___________
number of groups: >3
__anova__________ or
Discrete/Categorical
number of groups: 1
__chi-squared test__________
number of groups: 2 (intervention and comparator)
_chi-squared___________ or
Evaluating the RESULTS
Participants
Baseline characteristics reported in results section
Internal validity
Similar baseline demographics
Strength
External validity
Generalizable to
strrenghth
Consolidated Standards of Reporting Trials (CONSORT)
Comprises 25-item checklist and a flow diagram for transparent reporting of clinical trials (strength) - what happens to the people as they go through the trial
Flow diagram
Assess reasons for withdrawal
Impacts external validity
Rule of thumb
At least _80%___________ of the study participants should complete the study/be included in analysis (strength)
Endpoints
Statistical significance
p value
If p is < alpha, results are statistically significant
Confidence interval (CI)
If confidence interval does not include the value of no difference, results are statistically significant
Continuous endpoint: 0 = no difference
Dichotomous endpoint: 1.0 = no difference
Clinical significance
Continuous data:
Evaluate absolute difference between groups
Upper and lower ends of 95% CI
Dichotomous data:
Evaluate the HR/OR/RR
Calculate the RRR (1- RR), ARR (%intervention - % control), NNT/NNH
Upper and lower ends of 95% CI
Benefits versus risks
Consider: burden of administration, adverse events, cost
Compare to existing data
Evaluating the DISCUSSION and CONCLUSION
Discussion
Authors’ interpretation of results
Comparison to previous data
Strength and limitations of the study
Highlight unexpected results
Explanation of how results add to current data
Conclusion
No new data should be introduced
Concise overall summary of findings
Conclusion must be consistent with the data
Application
Impact of trial on clinical practice
Questions to ask about the trial as a whole:
What are the strengths?
What are the weaknesses?
Can the results be applied to the general population with this condition?
Will the results of this study change practice? Why or why not?
Does this trial provide strong/moderate/weak evidence for/against the intervention?
Consider
Statistical significance
Clinical significance
Internal/external validity
Feasibility of treatment
Resources to use when determining the application
Clinical expertise
Journal article editorial
Previous data
DynaMed
Medscape
Pharmacist’s Letter
Experts in the field
