Midterm Flashcards
Approximately ______ of underpowered clinical trials will actually end up being accurate and true, assuming they are not further weakened by biases.
¼
Heirarchy of evidence… Lowest to highest
o Anecdotal observation, “experience” and expert opinions o Animal model and “in vitro” studies o Case reports and Case series o Case-control studies o Cohort studies o Randomized controlled trials o Systematic reviews o Metaanalyses
The types of studies we see in clinical research are:
o Therapy studies o Diagnosis studies o Harm studies o Prognosis studies o Systematic reviews
The qualities of credible news and reporting sources include:
§ avoid sensationalized titles or teaser headlines
§ source the original citation
§ speak specifically to what can AND cannot be concluded from the data
§ offer context to the study and results
Signs you’ve gotten some “bad science”
§ Sensationalized/teaser headlines
§ Misinterpreted results
§ Conflicts of interest in study authors or media agency reporting the science § Misusing correlation to imply causation
§ Small sample sizes
§ Non-representative samples in study § Absence of controls or comparator populations that are similar to the study population
§ Absence or impaired blinding
§ Selective or partial data reporting
§ Irreproducible study design or results
§ Non-peer reviewed publisher of study
Clinical opinion should pass three tests:
o More generalizable evidence is not available
o The opinion has biological plausibility and incorporates related evidence
o Common sense
Case Studies/Series are valuable because they:
- Expose colleagues and others to new disorders or unique manifestations of known disorders
- Offer new insights or unique perspectives on diseases or their management
- Frame previously disparate, complex or disorganized understanding(s) in new ways that help organize what is known in more functional, useful ways
Case studies are limited because they do not control for:
- Placebo
- Temporal coincidence
- Natural history of disease
- Bias introduced by the patient, clinician, third parties, perception, etc.
Retrospective studies that start with two groups, one with outcome/disease of interest and one without, and review experiences and exposures of both groups looking for statistically significant differences in rates of exposure to a defined risk factor
Case control studies
- The comparator groups should be as similar as possible except for the disease (age, socioeconomic
status, gender, ethnicity, geography, occupation, etc.)
What kind of studies are typically case control studies?
Risk or harm studies (as opposed to therapy studies)
Ex: radiation exposure, chemotherapy
What kind of bias are case control studies prone to?
Recall and belief bias
Advantages of Case Control studies include:
- Less expensive
- Quicker and easier to conduct
- Can assess multiple exposure profiles - Particularly valuable for evaluating rare disorders
Disadvantages of case control studies include:
- prone to recall bias because they are retrospective
- Can only assess one outcome
- Cannot establish an accurate risk profile
- Cannot be used to establish prevalence
Why can case control studies not be used to establish prevalence?
you’re selecting folks on the basis of the existence of a disease, therefore you only have access to those we “know” are diseased… no greater
population to pull from to identify “how often” this disease occurs.
Prospective studies that identify two groups who are equal in every way except exposure to some risk factor, follow them forwards in time and calculate the rate of specific diseases in both groups
Cohort studies
What kind of bias are cohort studies prone to?
Attrition (drop out) bias - this could lead to high or uneven drop out rates in the different groups, which could impact validity
Advantages of Cohort Studies
- Prospective (usually)
- Can establish risk profile
- Can assess multiple outcomes/diseases
- Good for very rare exposures (e.g. nuclear plant melt-down, volcano eruption, etc.)
Disadvantages of cohort studies:
- Expensive
- take a long time
- hard to conduct (following exposed groups over decades is HARD)
- Not great for rare diseases
- Not good for diseases with very long latency periods
Study in which subjects are selected based on exposure to a risk factor. Their historical health data is reviewed
looking for health problems they’ve had or have
Retrospective Cohort Studies
A special variant of cohort study that looks to the past to examine medical events or outcomes.
What is the primary disadvantage of a retrospective cohort study?
data quality - the existing data may be incomplete,
inaccurate, or inconsistently measured between subjects.
Do retrospective cohort studies identify correlation or causation?
Correlation only
What is the difference between DDX Case Control vs. Retrospective Cohort designs
- Case-control studies select cases and controls based on the presence or absence of the OUTCOME
and look back in time for disproportionate exposure rates - they know the outcome and are looking for differences in risk factor
exposure rates - Retrospective cohort studies select cases and controls based on presence or absence of EXPOSURE to a risk and look back in time for disproportionate outcomes events - they know the risk factor exposures and are looking for
differences in outcomes (diseases)
What kind of study is this an example of? Why?
1000 patients with lung cancer vs. 3000 age, occupation, SES, ethnicity matched controls: health histories are reviewed on all to identify whether they were smokers. Results: Lung cancer occurs far more commonly amongst smokers than non-smokers.
Case control, because the outcome/presence of disease is known. They are looking for difference in risk factor exposure rates.
What kind of study is this an example of? Why?
1000 patients with a history of smoking vs. 3000 age, occupation, SES, ethnicity matched controls with no history of smoking: Rates of lung cancer are compared between the groups. Result: smokers’ risk of developing lung cancer is significantly higher than nonsmokers.
Retrospective cohort study, because risk factor exposure is known. They are looking for differences in outcome.
Type of study in which patients are selected based on disease status (i.e. all subjects have the disease) and divided into different treatment groups
Randomized Controlled Trial
What are the groups into which patients are divided in a Randomized Controlled Trial?
- Receive the experimental therapy
- Receive the “usual therapy”
- Receive a placebo therapy
- Receive nothing at all
What kind of study design is the best for therapy studies?
RCT
What type of study design is capable of establishing causation (not just correlation)?
RCT
What are the advantages of an RCT?
- quick
- inexpensive
- precise to conduct and analyze
- more easily combined with other studies in meta analyses and systematic reviews for better generalization of findings.
What are the disadvantages of a RCT?
- Generalizability of individual studies is limited
- Subject recruitment can be challenging
- Subjects must comply with their group’s assignment- this can be frustrating and result in
cross-over, cointervention seeking and/or drop out - Still not cheap, although less expensive
- Absence of an effective sham can weaken blinding and validity
When asking a quality clinical question in the musculoskeletal realm, some of the important outcomes to consider are:
- decrease in symptom severity,
- decrease duration or frequency of the symptoms
- improved function (or decreased disability)
- return to work
- decrease recurrence rates
- return to play and patient satisfaction
What does PICO stand for?
P - patient/population/problem
I - intervention
C - comparison intervention
O - outcome
Important sources of Bias include:
o Randomization & allocation concealment
o Blinding of patient/provider/outcomes evaluator
o Data analysis & reporting (selectivity?)
o Comparability of groups
o Similarity of co-interventions
o Compliance & follow-up rates
o Timing of the outcome assessments
technique used to prevent selection bias by concealing how participants are going to be assigned to intervention groups until the actual moment of assignment - used to implement the randomization sequence, not to generate it.
Allocation concealment
influencing (unconsciously or otherwise) which participants are assigned to a given
intervention group
Selection bias
The best allocation concealment strategies will include:
o Full disclosure of how allocation is done, including who had access to what knowledge and when
o Randomization scheme is withheld from the person who enrolls & assigns patient to a group
o Enroller is independent of caregivers and outcomes assessors
o Outcomes assessor is independent of caregivers and patient disclosure
Untrustworthy allocation concealment strategies will include:
o People who should not know have access to finding out or are able to guess who is assigned to which
group(s)
o Assignment scheme known or is predictable or detectable by anyone who should not know
o Incomplete Disclosure or the strategy is sufficiently sketchy that it would be reasonable to assume that
access was obtainable
BEST Allocation concealment is achieved by using ________.
a centralized off-site computer allocation process
Uncertain allocation concealment does ______
does not say one way or the other or does not clearly indicate how they did it
Low quality allocation concealment _______
allocates alternate subjects, allocates subjects to groups on alternate days as they first come in for treatment or selects subjects from databases or in another predictable way
the process of placing eligible persons into one of the groups in the study (i.e. experimental
group, alternate therapy group, sham or placebo group)
Randomization
Randomization accomplishes the following:
- Avoids selection bias
- Controls for temporal effects (groups have different average time affected by condition)
- Controls for regression to the mean
- Attempts, based on the laws of probability, to evenly distribute potential baseline characteristics that might give one group an advantage over the other
baseline characteristics that might give one group an advantage over the other
Confounders
Small study populations have a more difficult time achieving balanced randomization because ______ can occur
“mini-clusters”
I.e. larger populations studies are more likely to have a 50-50 distribution
What are the three main types of randomization?
- simple
- stratified
- block
Randomization achieved through the use of random number tables, tossing a coin, drawing different
colored balls, & drawing of ballots with the study group labels from a dark bag, etc.
Simple randomization
What are the advantages of simple randomization?
- simple
- when done with integrity (no peeking in the bag) it is difficult to go wrong.
Simple randomization can suffer from what kind of bias?
Chance bias
What is chance bias?
when randomization, by chance, results in groups which are not balanced in
important covariates or number of subjects. (Think coin toss)
Type of randomization that identifies covariates in the population and ensures that they are evenly distributed between treatment groups by separately randomizing those with the covarities
Stratified randomization
A type of randomization that can balance the numbers in each group at any time during the trial.
Block randomization
Ex: after a block of 10 participants we’re signed, 5 would be allocated to each arm of the trial
Ex: separate blocks for patient with a known comorbidity and those without
What is the most important reason for block randomization?
Guarantee balance of comorbidities between the various arms of the study but using separate blocks for patient with a known comobidity and those without.
Less important reason is for even number distribution
A method of generating equivalent groups by matching into pairs or triplets of patients based on covariates (age, disease severity, comorbidities, exposure to prior treatments like chiropractic, etc.) and then randomly allocating each pair to the intervention
Paired randomization
Type of allocation where groups are formed using an algorithm (read that as “computer generated”) that makes sure the groups are balanced.
Minimization
Can be superior to randomization for the formation of equivalent groups
Unacceptable methods of randomizing include:
o Alternating subjects o Birth date o Social insurance/security number o Date in which they were invited to participate o Hospital registration number
Keeping information about the test masked from the participants, to reduce or eliminate any behavior, belief or design that has reasonable potential to systematically deviate a result
from the underlying truth
Blinding
non-directional, equal opportunity offender of error
Random error
Error with directionality that leads to under or over-estimation of treatment effect (or harm)
Systematic error = bias
Blinding is NOT the same as concealed allocation. Why?
Blinding” occurs after allocation and is maintained throughout the trial
Who are the “big three” that are blinded in a study? What are others who could also be blinded?
The “big three” - providers - patients - assessors Others - statisticians - adjudicators - support personnel
What are the 4 potential points of group comparison?
- comparable at baseline/start
- comparable co-interventions
- comparable attention
- comparable compliance
In order for groups to be comparable at baseline, confounders should be evenly balanced after randomization. What are some examples of confounders?
- age
- sex
- race/ethnicity
- socioeconomic status
- type of condition
- duration of symptoms
- location of symptoms
- severity of symptoms
- type of work
What are some possible ways that comparable group co-interventions could be compromised?
- unscheduled extra help/therapy that was not permitted in the trial
- rescue medication
What does it mean to say that groups had comparable attention?
Groups were treated in a similar manner (# of visits, duration of visits, rotating practitioners)
What does it mean to say that groups had comparable compliance?
The groups evenly did and/or did not do as they were suppose to
What are some things we want to know about drop out?
- How many/how much was lost?
- How was the loss distributed?
- Evenly balanced across groups?
- Why was the data lost?
- Adverse effects? Got well?
- Were lost subjects similar to the subjects who stayed?
- When were they lost?
- Was it corrected for?
Why is it important to follow up on drop outs and understand that cause of their drops out?
- Subjects that dropout might be different than those that stay (different prognosis)
- Loss of randomization
- Loss of statistical power
- Hiding potential adverse effects
What is the 5-20 rule?
- drop out rate under 5 %: very limited risk of bias
- drop out rate over 20%: high risk of bias
- drop out rate between 5-20%: modest risk
At the end of a clinical trial, what are the two components of the ideal scenario?
we know the status of every patient with respect to the target outcome AND the follow up period was long enough for the study to be useful clinically.
What is immediate follow up?
As soon as one treatment is completed (0-48 hours)
What is short term follow up?
4-6 weeks
What is long term follow up?
Depends on the nature of the disease:
- 6-12 months (for LBP)
- Several years (for DJD)
After subjects are randomized, even
if they never received treatment, discontinued the trial or crossed over to the other group, what should be done with them?
They should be analyzed in the group to which they were first randomized to.
If randomized subjects never received treatment, discontinued the trial or crossed over to the other group, what are the ways that this could be dealt with?
- per protocol analysis
- worst case analysis
- intention to treat
- both worst case analysis and intention to treat analysis are done and compared
What is per protocol analysis?
Drop outs are tossed out of analysis - this is NOT ideal
What is worst case analysis?
- Drop outs from the group that did better are assigned negative outcome
- Drop outs from the group that did worse are assigned positive outcome
- This prevents overestimating effectiveness and in fact likely underestimates it
What is intention to treat analysis?
- The status of the subject at the time of drop out/crossover is forwarded to the end of the study for analysis
- this will underestimate the effective of a treatment
- the greater the number of dropouts/crossovers that are included, the less this compensated for lost data
What is the benefit of doing both an intention to treat analysis and worst case analysis of drop outs/cross overs?
If the results do not change when the outcomes of each analysis are compared, you can count on the effect being neutralized.
Is the results do change, the truth is likely in between the two methods’ outcomes
What are examples of X-factors that could be concerning as sources of bias?
- presence of conflict of interest
- unacceptable “therapeutic” dose used
- outcome measures used to judge success are in validated and/or inaccurate
- Flawed sampling process
- providers unequally and/or inadequately experienced
- assessors and therapists inconsistent in their individual performance and between others doing the
same job
Blinding is
the process of preventing individuals who have a reasonable potential to engage a behavior, belief or influence on the outcome of a study from having information that could reasonably allow that to happen.
