Exam 1 Flashcards
What is a placebo? What is it not?
Something that is inert (inactive, not therapeutic) but use it because of that placebo effect (the thought in the mind that it is doing something)
- It is not “nothing”
What two individual are accredited for the development of evidence-based medicine?
Dr. James Lind
Dr. David Sackett
Describe the relationship between Dr. James Lind and EBM
- accredited for discovering scurvy
- took 12 sailors, paired them, one got nothing and the other got oranges/lemons
- ompared both groups
- early version of a study
Describe the relationship between Dr David Sackett and EBM
- Father of EBM
- Established core principles of EBM in a landmark article in 1996
- Established 3 core principles:
a) Research
b) Clinical Expertise
c) Patient
Define EBM.
Evidence-based medicine (EBM) is the integration of best research evidence with clinical expertise and patient values
In EBM, what is a critical factor to evaluate effectiveness?
- Must be applied to the patient
- Although it may say in the guidelines, need to consider if it is useful for the patient
Why do we need EBM?
New therapies
New indications
New formulations
New “experts”
What are the five steps of EBM?
Ask: Develop your question
Acquire: Find the (best) solution
Appraise: critically evaluate the evidence for validity and usefulness
Apply: Use results in your practice
Assess: Evaluate your performance
In Step 1 (ASK): one should develop the __________
Research Question
A research question should be based on:
Population/Patient (Among….)
Intervention/Exposure (does…..)
Comparison (versus….)
Outcome (effect)
In step 2 (Acquire), one should ….. Examples?
Get the evidence - literature search
Medline, pubmed, cocharnae database, socialmedia, google scholar
In regards to a question about therapy, the ideal study type is:
RCT
In regards to a rsearch question about prevention, what is the ideal type of study?
RCT > Cohort Study > Case Control
In regards to a diagnosis question, what is the ideal type of study?
Prospective, blind controlled trial comparison to gold standard
In regards to prognosis, what is the ideal study design?
Cohort Study > Case Control > Case scenario/case report
For a research question on etiology/harm, what is the ideal study design?
RCT > Cohort Study > Case Control
For a cost analysis research question, the ideal study type is…..
Economic analysis
What often provides the best answers for clinical questions?
Meta-analyses and systematic reviews, when available, often provide the best answers to clinical questions
What is the benefit of a meta-analysis?
Meta-analyses and systematic review, when avilable, often provide the best answers to clinical questions
Describe the different types of research study designs
Study the graph
Describe the hierarchy of study design.
Study the pyramid
Define descriptive study. What is it simply?
One that is designed to describe the distribution of one or more variables, without regard to any causation or other hypothesis. (Just a snap shot in time)
Define analytic study.
Analytic studies test hypotheses about exposure- outcome relationships and measures the association between exposure and outcome throughout time.
How should one be able to distinguish and RCT?
Should be able to pick out RCT’s as should mention “randomization” in the methods
In step 3 (appraise), one should….
Critically Appraise: The process of carefully and systematically examining research to judge its validity, results and relevance.
Why is critical appraisal important?
Study results are not always:
Valid (actually due to the intervention/exposure)
Safe (invalid results can be harmful/wasteful)
Useful (meaningful or applicable to your patient)
Define internal validity
The degree of confidence that the causal relationship you are testing is not influenced by other factors or variables
Are the results due to the intervention (or exposure) we are studying… or something else?
What are the 3 major threats to internal validity?
Chance
Confounding
Bias
Define external validity . What does it depend on?
Are the results applicable (generalizable) to other populations (patients), settings and time?
Typically depends on study population (inclusion and exclusion criteria) and setting
Give an example of internal and external validity
Internal: Those who took Vitamin D scored 10% higher, is this due to the Vitamin D or something else
External: Depends on the exclusion and inclusion principles –> Can you apply those results of the Pharm.D. students to engineering students.
What is chance?
Random error
Inherent in all measurements (we are only taking a sample of all the possible observations)
The more there is, the less precise the results
What is an example of chance? Can you fully eliminate chance?
In the Vitamin D group, exam results were higher. Can I take those results and apply them to every pharmacy student in Canada –> NO
If we measured all pharmacy students in Canada, results are more precise; however, never fully able to eliminate chance.
The more measurements we take the more precise and closer to the actual results.
What is bias?
Any systematic (not due to chance) error in a study that results in an incorrect estimate of the association between the intervention or exposure and the outcome
…or…
Problems with the way a study was designed, conducted, or classified that leads to incorrect results or conclusions
What is confounding? What msut it be related to? Provide an example?
- A confusion of effects
- When some factor(s), other than the intervention or exposure under study, influences the observed outcome
- The factor must be linked to the exposure and the outcome
- Coffee causes heart attacks; however, coffee drinkers are more likely to be smokers. Therefore, may not be coffee at all.
External validity is related to…… What is an issue associated with external validity?
Related to the inclusion/exclusion criteria
Studies (especially RCTs) are very specific about who is and isn’t allowed in the study
Often becomes an issue with drug studies:
Who should use the drug?
Who the drug is approved for?
What is chance? Is it beneficial or detrimental to studies? How is chance calculated?
Random error – inherent in all measurement
Less random error = good
More random error = not good
Estimated and reported using statistics:
P-value
What does the P-value represent?
Probability representing the strength of evidence to support the null hypothesis
P value represents the probability the results are due to chance rather than a real treatment effect (want a low p-value)
What is the null hypothesis in studies? What is the alternative hypothesis?
Null Hypothesis: Set at showing no difference between the groups
Alternative Hypothesis: There is a difference between the groups
Describe the relationship between P-value, null hypothesis, alternative hypothesis?
Large P value (>0.05) = supports null hypothesis
There is no “statistically significant” difference between the groups’ results
Small P-value (<0.05) = reject the null hypothesis
There is a “statistically significant” difference between the groups’ results
How can one deal witch chance within a study?
Increase the sample size (not always possible)
Recognize the extent (through statistics) and interpret the results accordingly
Recognize that Statistical significance ≠ clinical significance (do not go hand and hand)
Is there a relationship between statistical significance and clinical significance? Example?
Statistical significance ≠ clinical significance (do not go hand and hand)
If a study concluded a 10 day difference in survival, this could be statistically signifiant but is not clincially significant
What are some examples of confounders?
- Smoking
- Age
- Exercise Level/week
- Blood pressure
- Diet
-Sex
Some may be hard to measure: stress, socioeconomic status, genetics
Can someone get some extra addictive dumauriers to smoke saturday
How can one deal with confounding?
1) Randomization (Gold Standard)
- Ensures groups are similar in all aspects (known and unknown factors equally distributed)
2) Stratification
- Stratify by a certain factor
3) Matching
- Observational studies (match by age and sex)
4) Statistical Models (multivariable models)
- “Controlled for” or “adjusted for” –> Attempts to adjust for confounding (only variable known and can measure)
How can one indicate confounding? What should we do?
TABLE 1: BASELINE CHARACTERISTICS
- use clinical judgement to determine if there is a difference between the groups
Importance of Table 1 (population characteristics):
1) Indicate confounding
2) Provides insight on external validity (gives insight on people studied and whether can apply to patients)
What is bias? What usually causes it?
Problems with the way a study was designed, conducted, or classified that leads to incorrect results or conclusions
Usually because the groups were treated differently somehow
What are the two main types of bias? Where do they come from?
1) Selection Bias
- Problems with how the study subjects were selected
- Want everyone to be selected from the same patient population
2) Information Bias
- Problems with measuring, collecting or analyzing information (exposure and/or outcomes)
Define selection bias? What does it effect? Types of selection bias?
Systematic error (or differences) in how the study subjects were selected or who participated
Primarily affects external validity
Self Selection/volunteer bias
Healthy Worker (adherer) Bias
Attrition Bias (lost to follow up)
What is self-selection/volunteer bias? What bias is it?
- People who volunteer/participate in studies are different from those who don’t
- Selection Bias
What is healthy worker (adherer) bias?
Employed (higher SES-socioeconomic status) individuals are usually healthier
What is attrition bias (lost to follow-up)? What is the main problem and when it is an issue? Why is this detrimental to a study?
- People who leave study e.g. a/e, found process to rigorous, drop out of pharmacy, death
- Main problem is when they drop out of one group (5 and 5 would be less concerned than 10 for 1 group)
- Becomes an issue if there are differences between those who were lost and those who weren’t
a) Lose important data (why did they drop out/withdraw?)
b) If a whole ton drop out of one arm, lose benefit of randomization (not equal anymore)
c) Chance becomes more of an issue
Define information bias.
Systematic errors in the way subjects were measured or classified
What are the types of errors that lead to information bias? Examples?
Outcome errors (RCT and observational studies)
-Problems with measuring tools (one electronic bp monitor, one group manual)
-Problems with actual measurements
Exposure errors (more so with observational studies)
- Problems with how subjects are categorized (non- vit. D group taking a multivitamin)
- Problems with measuring tools
Subject or observer variation (interpretation of pain) – everyone different and unique
When is information bias a concern?
Primarily a concern when the likelihood of being misclassified is unequal between groups
What are some examples of information bias?
Recall Bias
Interviewer Bias (Minimized by standardized forms, questions)
Surveillance/detection bias
Define recall bias
Individuals remember things differently
Subjects with the outcome (especially if negative) are more likely to remember their exposure – e.g. woman born with a baby birth defect (I did take that bp med one time, where as someone with a healthy baby would not)
Define interviewer bias? How can it be minimized?
Interviewer asks about exposure/outcome differently
Leading, probing or influencing questions
Multiple interviewers
Minimized by standardized forms, questions
Define surveillance/detection bias.
One study group followed more closely than the other
“If you look…you will find”
More outcomes identified because more follow-up, not necessarily because it occurred more in that group
What is a goal of studies in regards to bias?
Goal is to minimize bias when designing the study
Will never completely eliminate
Recognize and acknowledge it
Study the Graph on Slide 16
What are some strategies for reducing bias?
Clear definition of study and sample population
Ensure all groups are treated the same except for the intervention
Standardized measurements
- Same questionnaire for all subjects
- Same interviewers
- Automated devices
- Centralized labs (all analysis in same labs)
Collect the same information from all subjects in the same way
Blinding (pt or researcher or both are unaware of what group a t is)
What is publication bias? What does it effect? Example?
Authors and journals tend to publish positive findings, especially with drug trials
May lead readers to think there is a consistent association when it’s not really the case
Not effecting internal validity
E.g. Trial A finds that a new drug is effective (publish), Trial B shows significant adverse effects (not publish) potentially giving a misleadingly favorable impression of the new drug’s effectiveness and safety
What is a randomized controlled trial?
Test whether an intervention works by comparing it to a control condition
How are subjects divided in an RCT?
Subjects assigned to a study group:
Intervention group
…or…
Control group (no intervention, alternate intervention)
In an RCT, what does it mean to be randomized? What groups are randomized?
Randomized = subjects assigned to study groups randomly
Have an equal probability of being assigned to any group
Can be more than 2 groups
Randomization is critical for…..
Minimize confounding
Can randomization occur in an observational study?
- No. Cannot occur in an observational study.
What are the different types of RCT types?
Parallel
Cluster Randomized
Cross-over
Describe a parallel RCT study design?
A study sample is selected from a population. From the study sample, subjects are further divided into an intervention or control/comparison group. These groups are followed over time and monitored for the outcome.
Describe a clustered-randomized RCT study design?
Clusters of individuals are randomized instead of individuals (e.g. cities, classes, pharmacies, hospitals)
A cluster randomized trial (CRT) is a randomized controlled trial in which pre-existing groups, called clusters, of individuals are randomly allocated to treatment arms.
When is a cluster-randomized trial used?
- Most commonly used when studying an approach to patient care
Describe a cross-over RCT?
A study sample is selected from a population. From the study sample, subjects are further divided into an intervention or control/comparison group. These groups are followed over time and monitored for the outcome.
There is then a washout period in both study arms. After the washout period then the groups are flipped.
Intervention Group –> Control group
Control Group –> Intervention Group
We then follow the groups over time and monitor the outcome.
The outcomes are then compared. One is acting as their own control.
What is a beneficial characteristic of a cross-over RCT? What is the limitation?
- The individuals act as their own control and therefore confounding is effectively minimized
- This type of study does not require as many subjects
- It is a more complicated study to run and the wash out period can be lengthy
In regards to selecting an RCT design, what is a critical rule to follow? Why is this rule critical?
The study must be designed before it has begun
- This limits bias and maintains integrity
How is the research question established for an RCT?
Population – who’s going to be included?
Intervention – what are we testing?
Comparison – what are we comparing it to?
Outcome – what outcome are we looking for?
When selecting a population for an RCT, what needs to be established?
1) Start with a defined population
- Who we want to study
- Where do we get them from (all subjects should be from the same population)
- e.g. Hospitals, clinics, advertisements
2) Specific inclusion and exclusion criteria
- Much more strict in RCT
- Can limit generalizability
3) Sample Size (n)
What is the disadvantage of having specific inclusion and exclusion criteria?
Can limit generalizability
Define sample size (n). How is the sample size (n) determined?
Sample Size: how many subjects are required
- This is determined by the study power
Study power = ability to show a difference between groups if a difference really exists
n too low = underpowered
n too high = unnecessary exposure, more resources
Optimal number is determined statistically (studies recruit around this power number)
Must be determined BEFORE the study starts
Define study power
ability to show a difference between groups if a difference really exists
An increase in a study size will diminish…..
CHANCE
In an RCT, what is the intervention?
The exposure of the experimental group. Must be specific.
If it is a Drug must include:
Dose
Regimen
Delivery method/Formulation
Follow-ups
Length of intervention
If it is a Procedure (e.g., pharmacist counseling): must include
What’s involved
Timing
Follow-up
Length of intervention
In an RCT, what is the comparison (control group)? What can it be? What should we consider?
- What we are using to compare our intervention to
Nothing (placebo/usual care)
A different dose
A different drug/procedure
Standard treatment
- We need to consider how many controls will be used
In regards to the controlof an RCT, what is critical to ensure?
Controls must be treated exactly the same way as the intervention group – the only difference is the intervention
In an RCT, how is the outcome of a research question divided?
- Could be anything but must be clearly stated and logical
- May be objective or subjective outcomes
- Established hard or surrogate endpoints
- Established primary endpoint or composite endpoint and may have potential secondary endpoints
In an RCT, the outcome may be _________ or _________. Define.
Objective
Measurable (e.g., blood pressure, lipid levels)
Subjective
Subject’s interpretation (e.g., back pain)
In an RCT, endpoints may be _______ or _______. Define.
Hard endpoints - Often used as big clinical outcomes
E.g., Death, stroke, MI
Surrogate endpoints - Used as a proxy or estimate of the hard endpoints
E.g., Blood pressure (…which can lead to stroke)
In an RCT, what is a primary endpoint
Main result that is measured at the end of the study to see the effect of the intervention
In an RCT, what is a secondary endpoint? Disadvantages and Advantages?
Additional results of interest, but not the main focus
Caution when interpreting as the study wasn’t designed around them (statistical issues)
NEVER make a decision based solely on a secondary endpoint
Allows for the formation of a hypothesis for future studies
What is a composite endpoint? What is a classic example? What is the advantage? What is the disadvantage?
A primary endpoint that contains several events
E.g., MACE – Major Adverse Cardiovascular Events Instead of just looking at death as a primary outcome, you include MI, stroke and death together
Benefit = less subjects needed
Disadvantage = hard to determine the true effect of the intervention on each of the event types
- Must Look at specific results for breakdown
Randomization helps to reduce…..
Confounding
What makes RCT’s the gold standard for study designs?
Randomization
What is the goal of randomization?
We want the groups to be as similar as possible, except for the intervention
Randomization (if done correctly) will ensure that the groups are similar in ALL aspects
Similar in things we can measure (E.g., age, sex, blood pressure)
Similar in things that are unknown (E.g., genetics)
How may randomization be conducted? What is inappropriate randomization? What is a rule of the randomization PROCESS?
Coin toss, table of random numbers, computer generated, etc.
NOT by days of the week, alternating subjects, etc.
Must have an equal chance of being in either group
and should not be able to tell which group will be assigned next
What are the main methods of randomization?
Complete
Block
Stratified
What is complete randomization?
No limitations – straight up randomization
Won’t necessarily get equal numbers in each group
What is block randomization?
Used to force balance in the number of subjects in each group
E.g., block of 6 = for every 6 subjects randomized: 3 in intervention group, 3 in control group
Is a researcher involved with randomization?
NO
Researcher is not involved with randomization; someone outside the study to minimize bias
What is stratified randomization used to achieve? What are some examples?
Used to achieve similarities in certain baseline characteristics
E.g., stratify by age = everyone under 65 is randomized and everyone over 65 is randomized
E.g., stratify by clinic = randomization is done at
each individual site
Where does randomization occur?
Usually randomization is centralized and offsite
What is allocation concealment?
Hides the sorting of trial participants into treatment groups so that this knowledge cannot be exploited
Refers to not disclosing the allocation (or randomization) scheme to researchers or participants.
Critical to the randomization process of an RCT
In an RCT, is allocation concealment the same as blinding?
NO
Allocation concealment differs from blinding in that blinding is not disclosing the treatment allocation AFTER randomization has occurred.
Give an example of allocation bias?
Blue Pill vs Red Pill in preventing death
- If the researcher knew the randomization scheme, they may wait to randomize a particular individual into the intervention arm as they know they have a higher chance of death
- Meredith Grey
What is the benefit of allocation concealment?
- helps to reduce selection bias and allocation bias
Allocation Bias: you can figure out the randomization scheme, so put patients into a specific group to get a certain treatment
Define intention to treat (ITT). How can one tell if this was done?
Analyzing the data for ALL patients and according to the group they were originally randomized to
“Lost to follow-up”: Drop-outs, switch groups, deaths, etc.
- Need to include these people in our analysis. Baseline and results should match.
Define “per protocol”
Analyzing data only from subjects who completed the study or followed protocol exactly
What is the benefit of ITT?
ITT preserves the value of randomization (might be differences b/w those lost and those remaining)
Lots of important data in those lost to follow-up (adverse events)
What is blinding?
Unaware of what study group a subject is in
If one was to know an individual’s group assignment, it could cause:
Subjects may report outcomes or adverse effects differently or behave differently
Physicians/investigators may report outcomes or events differently or treat subjects differently
Researchers may collect data, classify outcomes or events and interpret subject response differently
Affects the internal validity of the study - BIAS
A lack of blinding effects what:
Internal Validity - BIAS
What types of RCT’s blinding is there? What is the most common?
Unblinded (open-label) –> Sujects, researchrs and evaluators are not blinded
Single blinded –> One group blinded (often the evaluators)
Double Blinded –> STANDARD Often subjecys and researchers blinded (may see evaluators blinded)
Triple Blinded –> All 3 groups blinded
When does blindning become less critical?
The more objetcive the outcome is, the less critical blinding becomes (e.g. stroke vs pain severity)
What does blinding reduce?
- Blinding reduces bias
What is the placebo effect? How does this occur?
Placebo – Latin “shall please”
Perceived or actual effect from an ineffectual or inactive treatment
Conscious belief about drug/treatment
Subconscious association between being “treated” and recovery
Can be positive (benefits) or negative (adverse event)
Sugar pill, saline, sham procedure
Effect of cost, “severity” of treatment…
Define observational study?
Observe the effect of an exposure (or intervention) on an outcome without “interfering” with anything
Exposures: risk factor, drug, environmental, procedure, etc.
What does an observational study set out to achieve?WHy is that?
Looks for associations (not causation)
Can not prove cause and effect
To prove cause and effect, need strong internal validity –> Does not have the randomization piece
Randomization minimizes confounding (primarily, as randomizes people into groups which allows us to distribute factors that are known and unknown)
What are the benefits of an observational study?
Often more:
Practical / Feasible
Ethical
Real-world
–>Wider inclusion criteria (more generalizable)
–> Real-life situations (comorbidities, drugs, etc.)
Why may one use a observational study over an RCT?
Because clinical trials evaluate such a small percentage of people who will eventually use a studied drug and are of relatively short duration, it is possible that unexpected or rare drug effects will not be detected during the investigative process
What type of study’s help guide clinical decisions?
RCT’s
- Cannot use observation studies to guide lcinical practice
Obersavational study thrests? Internal validity?
Is the association real…or due to something else?
Is there internal validity?
3 major threats
Chance
Confounding
Bias
What are the types of observational studies?
(Case reports / Case series)
Cross-sectional
Case-control
Cohort
Define case report? What encompasses a case study?
Describes and interprets an individual case
Usually written as a narrative (e.g. story)
No predefined methods
No predefined outcomes
Patient described in detail
Observations described in detail
Exposure and outcome(s)
What does a case study usually depict?
Unexpected events and adverse effects
Important variations of known diseases
Weird observations that can’t be explained by known disease
What are the advantages of a case study?
Useful in identifying new or unusual trends or diseases
Useful in identifying new drug effects (good or bad)
Describe novel interventions:
–> Treatment
–> Diagnostic procedure
Suggest areas for further research (or “alarm”)
What are some disadvantages of case studies?
Difficult to interpret
–> Spontaneous appearance and disappearance of signs/symptoms
–> Placebo effect
–> Other influences
Can’t confirm or prove anything
What is a case series? What is its effect?
Collection of individual reports which typically occur within a fairly short period of time
Can be hypothesis forming…or world altering
What is a cross sectional study?
Looks at population or study sample at one point in time
Snapshot of what is happening at that time
What does a cross sectional study collect information on?
Exposure
Outcome
Health status
Health behaviours
How can one collect information in a cross-sectional study?
Surveys / Questionnaires
Phone, mail, online
Chart reviews
Administrative data (governments, insurance agencies)
What is an advantage (s) of cross sectional studies? What is a disadvantage?
Useful for describing prevalence
Proportion of a population or sample that has a condition (disease, risk factor)
E.g., The prevalence of MS in SK is 314/100,000
Can’t determine the incidence
Number of new cases (people) with a condition over a specific time
E.g., The annual incidence of MS in SK is 16.5/100,000
Can be analytical (looking for associations)
–> Factors that might be associated with one another
–> Can’t determine if an exposure causes the outcome though
What is are the advantages of cross sectional studies?
Good for determining prevalence
Useful for quick examination of potential associations
Hypothesis generating…future research
Inexpensive
What are disadvantages of cross-sectional studies?
Limited to what we know about the potential associations
–> Did the exposure precede the outcome?
–> Other contributing factors?
–> Transient effects
What is a case control study?
Retrospective (looking back in time)
Compares 2 groups:
Cases = people with the outcome
Controls = people without the outcome
Compare the proportion of cases with the exposure to the proportion of controls with the exposure
Important consideration in case-control selection?Why is this important?
Try to have the cases and controls as similar as possible except for the outcome (minimize confounding)
Describe the process of case-control study?
Have cases and controls in present
Look back in time. Analyze to see whether the subjects in both arms had an exposure (e.g. drug)
Compare the rates
What are the guidelines for selecting cases in a case-control study?
- Need to be very specific about the outcome
–> Want a “validated” outcome
–> It really is the outcome we think it is (e.g., actually had angina and not just chest pain)
–> Use standardized, accurate measurement/tools, criteria, definitions, etc.
Ideally incident (new) cases over prevalent cases
What is the difference between prevalent and incident cases?
–> Prevalent cases – don’t know when the outcome occurred, and if the exposure preceded it (has came after the outcome has occured)
–> Incident cases – we know when the outcome occurred and can be sure the exposure preceded it
How should one select the controls for a case-control study?Why?
Want the exposed and unexposed groups to be as similar as possible (except for outcome)
–> Should come from same population as cases (e.g. not having control from Toronto, and case from Saskatoon)
Often use multiple controls for each case
–> Increase sample size….decrease random error
–> Increases the power of the study (but no benefit after more than 4 controls per case)
What is the average number of controls to cases for a case-control study?
4:1 = 4 controls for every case
How can we make sure the cases in case-control are similar to the controls?
Often use matching to help ensure groups are similar
Helps decrease confounding (known confounders)
Can only match on factors we know (or can measure)
Sex, age, location, baseline cholesterol levels, etc…
What are the advantages of a case-control study?
Good for studying rare diseases or outcomes
Relatively quick to do since outcome has occurred already – data is already available
Good for establishing an association (leads to further research)
What are the disadvantages of a case-control study?
Not randomized – susceptible to confounding
Susceptible to bias
Only suggest associations – can’t prove cause and effect
What is a cohort study?
Groups (or cohorts) of subjects are created and compared according to exposure
Exposed vs. unexposed group
Groups followed over time to see if outcome occurs
A cohort study is similar to….
Similar to an RCT but we didn’t randomize subjects into group or assign the exposure
Why are cohort studies beneficial?
Best type of observational study for finding a valid association
What is the best type of observational study for finding a valid association?
Cohort study
What are the two types of a cohort study?
Propsective
Retrospective
Describe a prospective cohort study?
At the start of the study, outcome hasn’t occurred yet
We have an exposed and unexposed group. We follow over time and identify the number of people who experienced the outcome over time.
Compare the rates.
Describe a retrospective cohort study?
Outcome occurred before study started (uses retrospective data)
Start at a point in history with those exposed and those unexposed. Follow up to a a certain date. Analyze the number of people in both groups who have the outcome.
Compare the rates.
Contrast prospective and retrospective cohort studies?
Prospective:
Conducted in real-time
Have some control over what data you want to collect and where it’s coming from
–> Pre-existing data sources (charts)
–> Subjects (interviews)
Retrospective:
Good for long follow-up studies (observation is already complete)
Use retrospective data collected prospectively
Data limitations
Accuracy or completeness
Missing information on important factors (e.g., lifestyle) – Big Gaps
What is an exposure in a cohort study?
Drug
Procedure
Occupational
Environmental
Rules of a cohort study
Exposure and outcome must be specifically defined
E.g., degree of drug exposure (dose and length)
Exposure must precede the outcome
In an observational study, what is a benefit over RCT’s?
Chance is not as a major in the studies –> ONE THREAT MINIMIZED DUE TO LARGE NUMBERS
The numbers we get is significant as compared to an RCT so have less issues with chance
How should a cohort be selected?Howdo we do this? How many comparator groups?
Groups (exposed and unexposed or “comparator”) should be as similar as possible
Select from same population
Can have more than one “comparator group”
E.g., Various levels of exposure
–> Potential unknown confounders even if groups are similar
What are some advantages of cohort studies?
Provides highest level of evidence for observational studies
Can study multiple outcomes after a single exposure
What are some disadvantages of cohort studies?
Susceptible to bias
Not randomized so susceptible to confounding
More resource intensive than other observational studies
Define systematic review
A review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research
Collect and analyze data from the studies that are included in the review
Define meta-analysis
The use of statistical techniques (usually in a systematic review) to integrate the results of independent studies
What are the two steps of a meta-analysis
Extract the data and main result from each individual study
Calculate a pooled average result
–> Greater weight given to studies that provide more information (e.g., studies with bigger sample sizes)
What is the relationship between meta-anlyses and systematic reviews?
All systematic reviews are not meta-analyses and vice versa
Shouldn’t always statistically combine study results into one overall result
Why can’t you combine study results into one overall result?
Clinical reasons
Do all the studies address the same question?
Similar populations?
Similar interventions and/or comparisons?
Similar outcomes?
Statistical reasons
Study “heterogeneity” (how different are the studies)
Are the results between studies consistent?
Q or I2 statistic
WHy are meta-analysis and sytematic reviews done?
Tons of research
Systematic reviews provide opportunity and good summary of all clinical evidence out there
Who does systematic reviews or meta-analyses?
Anyone can publish a systematic review and/or meta-analysis
~2500 SRs indexed annually on MEDLINE
Often found in specialty journals (increasingly more seeing them being published in medical journals)
What is the cochrane collaboration?
1993 - Archibald Cochrane
Cochrane Collaboration
a compilation of high-quality, evidence-based medical research and systematic reviews
International network of 30,000+ volunteers in 53 review groups
Prepare, update and promote Cochrane Reviews
Internationally recognized as the highest standard in evidence-based healthcare (IMPORTANT)
No industry money
Why are cochrane reviews so valued?
Robust/consistent and specific methodology for review
No industry money
Outline the componenets of a systematic review?
Question of interest
–> Study protocol – inclusion/exclusion criteria
–> Decide if only RCT or also include observational studies
Comprehensive literature search – search terms (librarian involved)
Identify appropriate studies for inclusion
Go through abstracts
Scan the bibliography, reach out authors, research ongoing trials
Assess study quality
Extract data
Analyze data (if doing a meta-analysis)
Interpret results – write review
Questions can inform answers even although idiot
When doing a review? How mnay people involved and in what steps?
Identify appropriate studies for inclusion
Go through abstracts
Assess study quality
Extract data
Should be done by at least 2 separate reviewers
3rd reviewer to break a tie
What are some issues with meta-analyses?
Not all meta-analyses are done properly
Does the review address a clear question?
Did the authors look for appropriate papers?
Did the authors assess the quality of the included studies?
If there was a meta-analysis done, was it appropriate to do so? (Was a measure of heterogeneity reported? Q or I2 statistic?)
What is the overall result of the review?
Do these results apply to my patient(s)?
Who paid for the study?
What is a critical part of systematic reviews?
Garbage in = garbage out
Including results from poor quality studies
What is grey literature?
thesis, abstract, non-peer-reviewed (risks to this, not a necessity, useful if dealing with rapidly changing fields)
What are clinical practice guidelines?
Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific circumstances
What is the main point of a clinical practical guideline?
Get evidence into practice
What do clinical practice guidelines define?
Describe a range of practices for the diagnosis, management, or prevention of specific diseases or conditions that meet the needs of most patients in most circumstances (NOT ALL CIRCUMSTANCES OR PATIENTS)
What are the components of a clinical practice guideline?
Systematic review (+/- meta-analysis) of research
Focus on the strength of evidence (KEY) Need strong evidence
Set of recommendations
How to manage patient and/or condition
Describe target population
Should be specific
Inclusion and exclusion criteria
Describe target audience
Specialists / GPs
Other HCP
Patients
Administrators / Insurers
Expert Panel
Description of recommendation development
Actual methods
- Systematic review / Meta-analysis –> ESSENTIAL
- “Expert opinion” and “usual practice” SHOULD BE MADE CLEAR if lack of evidence for the opinion
External Review
Funding Statement
Some people always eat rice everyday for fun
The expert panel should be comprised of:
What should be mentioned about them?
Range of experts should be used (such as):
Specialists / GPs
Nurses
Pharmacists
Other HCPs
Patients (SHOULD BE INCLUDED – family members, people caring for them) Important for asking questions, patient values and interests
Statisticians
Librarians
Ethicists
Information on each panel member (including conflicts of ineterst)
What encompasses an external review?
Should be reviewed by individuals not involved in the development
Often sponsored or endorsed by organizations
Doesn’t always mean quality
How often are CPG updated?
Most guidelines only produced every few years
Quicker in fast-moving fields
(Should state the next date at which the guidleines are to be reviewed)
What are interim guidelines?
Focus around a specific clinical questions
Are subject to further updates, revisions, or developments
Often issued when there’s an urgent need for guidance in a particular area
(just a singular question)
A funding statement should include:
Should report where funding came from
Funder(s) should have no influence on the content
CPG recommendations should be:
Clear
Specific
Linked to evidence
How can one determine the strength of a guideline?
Most use the GRADE system
(Grading of Recommendations, Assessment, Development and Evaluation)
Number (1 or 2) = strength of recommendation
Letter = quality of evidence supporting the recommendation
What is the grading scale?
Study slides
What is a critical thing to remember when interpreting guidelines?
Garbage in = garbage out
Studies of poor quality
Poorly done systematic review / meta-analyses
What are some problems with CPG?
Poor quality studies (primary literature)
Incorrect or invalid information
Poorly conducted systematic review / MA
Publication bias
Self-serving / biased – drug company involved promoting their drug
Not user-friendly – readable and accesbible
Liability?
How can one critically appraise a practice clinical guideline?
AGREE II
23 items – 6 domains:
What are some barriers to guidelines?
Ultimate goal: want healthcare providers to use the guidelines
Barriers to healthcare providers using guidelines
Should be accessible free online
Guidelines should have summaries, decision aids, easy snapshots and tools to help apply the evidence
