Exam 1: Hoff Flashcards
Modern Definition of EBM
USe of mathematical estimates of risk of benefit and harm, deried from high-quality research on population samples, to inform clinical decision-making in the diagnosis, investigation, or management of individual patients
EBM emphasizes… (4 things)
Best Evidence
Clinical Expertise
Patient Values
Patient Circumstances
Best Evidence
Valid and Clinically Relevant
Clinical Expertise
Clinical skills/experience to understand patient’s situation, diagnosis, risks, and possible benefits from interventions
Patient Values
Preferences, concerns and fears, expectations
Patient Circumstances
Individual status and clinical setting
What are the 5 basic steps in EBM?
- Ask answerable questions
- Search for best evidence
- Critical appraisal for validity and relevance
- Integrate evidence, clinical expetise, and patient values/preferences and apply
- Evaluate results
Clinicians Incorporate Evidence by: Doing Mode (Define)
Carry out at least the first 4 steps (Ask AQ, Best Ev, Appraisal, Integrate/Apply, Evaluate)
Using Mode (Define)
Restricts searches to sources have already been critically appraised (only step 1 and 2)
***Generally for uncommonly seen problems***
Replicating Mode
Following the opinions of recognized experts regarding very rare entities
What does PICO stand for?
Patient or Problem
Intervention
Comparison Intervention
Outcomes
Background Questions
Seek General Knowledge (Students)
Two Components:
- Who, what, where, when, why, how?
- Plus the disorder itself
Foregroud Questions (Majority)
***Clinicians***
Seek specific knowledge for patient management
-Comprise three or four elements (PICO)
What source is the best when searching for evidence?
Electronic (online) sources
-More likely to be current, updated, and (now in many cases) evidence-based
What constitutes best evidence?
Current, valid, and clinically relevant
Hierarchy of study reliability:
Systematic reviews (or meta-analyses) of RCTs
Randomized, controlled trials (RCTs)
Prospective Studies
Retrospective Studies
Cross-sectional Surveys
Case Series
Case Reports
***SRPRCCC***
What is the best site for systematic reviews?
The Cochrane Collaboration
What type of study design is not randomized?
Case-control studies
Which type of study will bring bias, as certain things are being looked for?
Cross-sectional surveys
What is IMRAD and what is it used for?
Abstract
Introduction (why the research was done)
Methods (how the research was structured)
Results (what was found)
How Analyzed
Discussion (what the researchers think the results mean)
IMRAD is the standard structure of research papers
What 3 preliminary questions should be asked?
- Why was the study in question done; what hypothesis was tested?
- What type of study was done? (e.g. primary or secondary)
- Was the study design appropriate to the broad topic of research addressed?
What are examples of primary studies? Secondary studies?
Primary: Experiments, Observations, Clinical Trials (Therapies, Diagnostics, etc.), Surveys, Questionnaires
Secondary: Reviews (systematic or non-systematic), Economic analyses, Decision analyses
Intervention (Therapy)
- Best Study Design
- Description
- RCT
- Subjects randomly allocated to treatment or control and outcomes assessed
Harm/Risk/Etiology
- RCT
- Description
- RCT
- Similar to intervention questions, but RCTs to assess harmful outcomes are not ethical
Harm/Risk/Etiology
- Cohort Study
- Description
Outcome(s) compared for matched groups with and without exposure or risk factor (prospective)
Harm/Risk/Etiology
- Case-Control Study
- Description
Subjects with and without outcome of interest are compared for previous exposure (retrospective)
***more bias, but occasionally done to look at harm or risk***
Diagnosis
- Best Study Design
- Description
- Diagnostic Validation Study
- Independent, blinded comparison with reference (“gold”) standard
Prognosis/Prediction
- Best Study Design
- Description
- Cohort Study
- Long-term followup of representative cohort to determine outcome
Randomized Control Trials
- Concealed allocation
- Randomization generally should be concealed
Randomized Controlled Trials
- Single- vs. Double-Blind
Single (only subjects)
Double (subjects and investigators)
RCTs
- Advantages
Allow rigorous evaluation of a single variable
Designed prospectively
Seek to confirm a null hypothesis
Allow for meta-analysis
Minimize bias
RCTs
- Disadvantages
Expensive
Require long-term study (months to years)
Could introduce hidden bias
When is randomization inappropriate?
When the study involves prognosis of a disease
Investigating the validity of a diagnostic or screening test
Investigating quality of care issues when criteria for success are not known yet
Cohort Studies
- Design
- Prognosis Studies
- Bias
-
Observational - NO intervention
- Two or more subject groups selected based on exposure or no exposure to a particular substance/organism to compare outcome -
Prognosis Studies: special case - usually intended to determine what happens to people with a certain diagnosis or problem (e.g. heart failure) over time
- Require long-term follow up - Can be substantially bias
Case-Control Studies
- Design
- Bias
- Concerned with
-
Observational, not randomized
- Patients with certain condition, “matched” with controls data usually obtained retrospectively for exposure to a disease-causing agent or circumstance - More prone to bias than cohort studies
- Generally concerned with harm or etiology
Cross-Sectional Surveys
- Design
- Bias
- Other uses
- Representative sample interviewed, examined, or evaluated about a specific question
- Data are collected at a specific time but often include retrospective information - Potential for significant bias
- Also used in questions of Etiology
Case Reports
- Design
Simple report of a medical history
- Sometimes a series of patients’ histories are reviewed and analyzed together
- Weak statistically but may give occasional insights into rare or unusual conditions
Weakest: report of a single case
Systematic Reviews
- Design
- Key Features
- Commonly employed in evaluating/comparing RCTs; can be used to compare others as well
- Conducted according to strict methodology - Key Features:
- Statement of objectives, materials, and methods
- Include all original reports available, globally, even if unpublished
- Individual reports each critically evaluated
- Conclusions based only on studies meeting pre-set quality criteria
Systematic Reviews
- Advantages
- Large amounts of information assimilated quickly
- Explicitly limit bias when selecting studies for review
- Compare studies for consistency and generalizability
- Inconsistency (heterogeneity) is easily identified and new hypotheses can be formulated
- Conclusions are more reliable
- Quantitative reviews (Meta-Analyses) increase precission of overall results
Meta-Analysis
- Design
A statistical synthesis of numerical results of several studies which all addressed the same question
- Incorporate the advantages of systematic reviews
- Powerful statistical analysis
Diagnosis Studies
- Uses
- Randomization
- Screening a general or specific population, Diagnosis in an individual, Patient follow-up and management
- NOT randomized
Three Preliminary Questions to ask for all papers:
- Why was the study done? (purpose or hypothesis? background?)
- Primary vs Secondary
- Appropriate Design (e.g. RCT, Cohort, Case-Control, etc.)
***Check the abstract first***
What should be assessed first when reviewing the results of Diagnosis Studies?
Validity
Questions to ask when verifying Validity?
- Research question clearly defined?
- Presence or absence of disorder confirmed by reference test (aka the Gold Standard)
- How did investigators know the disorder was present?
- Was comparison indepndent and blinded? - Was test evaluated on an appropriate spectrum of patients?
- Reference standard applied to all patients?
-
Diagnosis Studies: Clinical Important
- Questions to Ask
- Ideal Test
How accurate is the test/What is its sensitivity and specificity?
***Ideal test will produce a high proportion of true positives and true negatives
What is the usefulness of any Diagnostic Test based on?
Its accuracy in identifying the disorder
Diagnostic Studies: Likelihood Ratios
Measure of accuracy
Predict likelihood of a certain result in a patient with the target disorder when compared to the likelihood of the same result in someone without it
Diagnostic Studies: Sensitivity vs Specificity
Sensitivity: proportion with** the disease testing **positive
Specificity: proportion without** the disease testing **negative
Diagnostic Studies: SpPin and SnNout
SpPin: if test with high specificity is positive, rules the condition IN
SnNout: if test has high sensitivity and is negative, rules the condition OUT
Diagnostic Studies
- Pre-Test Probability
- Post-Test Probability
- Pre: based on estimates of prevalence (how often a condition is found in a specific population)
- Post: a good test increases post-test probability significantly
Diagnostic Studies
What does a likelihood ratio (LR) of 1 mean?
LR > 1?
LR < 1?
LR = 1 means that post-test and pre-test probability is the same
LR > 1 increases the probability; says the test is accurate and helpful
LR < 1 decreases the probability; says the test is not helpful
***Represented with Bayes’ Theorem***
Diagnostic Studies
- Will the results change your management?
If the LRs are close to 1, then post and pre test probabilities will be similar and therefore not very useful
Therapy Studies
- Two key questions to ask
Do they have a clear question?
Is it randomized?
Therapy Studies
- About
***Most common kind of clinical papers, by far***
- Ideally performed randomized
- Studies of adverse reactions to therapy (studies of harm or risk) are done differently
Preliminary Questions for all papers
Why was the study done?
Primary vs secondary?
Appropriate design?
Primary vs Secondary
Primary: trying to find new info
Secondary: meta analyses or systematic reviews
Therapy Studies
- Validity
Clearly defined question?
Randomized?
***Blinded***
Concealed Allocation
Therapy Studies: Validity
- What is the most important thing to look for in RCTs?
Concealed Allocation
-Avoids selection bias if clinicians do not assign gropus
What is the Rule of Thumb for assessing the Validity of Therapy Studies?
If less than 80% completed followup, likely invalid
Therapy Studies: Validity
- Intention to Treat
Preserves randomization (minimizes bias)
-Means all data from subjects in the intervention group is analyzed
Therapy Studies: Validity
- What happens if researchers fail completely analyze subjects with their assigned group?
Results can be skewed toward the intervention
Therapy Studies: Importance
- Three questions to ask
How large is the effect of treatment? (accuracy)
How precise are the findings?
How useful is this in practice?
What is another term for systematic error?
Bias
What does a p-value of < 0.05 mean?
It means that there is less than a 1 in 20 probability that the result is due to chance alone
Smaller p-value = less likely due to chance
Therapy Studies: Importance
- Confidence Intervals
Stated in Results section of a paper as a range of values around a demonstrated difference
When the CI crosses 1.0 (“Line of No Difference”) the result of interest is not statistically significant
Therapy Studies: Importance
- Number Needed to Treat (NNT)
On average, if you do this intervention to this set of people who have this set of problems, if you treat x number, you will get a good result
***Number of patients you would have to treat to achieve one good result*** (or avoid one negative)
***Lower NNT = BETTER***
1 is the best, 100% will have an effect
Therapy Studies: Importance
- What is the most important part of treatment?
It is important that treatments improve outcomes that are important to patients
Better Approach: Is there a compelling reason why results should not be applied to your practice?
Summary of Therapy Studies
- Best Studies of Treatment
- Determine Validity (5)
- Determine Importance (3)
- RCTs
- Clearly defined research question
Randomization and blinding
Intention to treat
Equal treatments of groups
Comparable groups at the beginning
- Accuracy
Precision (p values, ***CIs*** (Best))
Usefulness in your practice
Harm/Risk/Etiology Studies
- Study Types
- Definition
- Cohort Studies (prospective) or Case-Control Studies (retrospective)
- Observational and Collecting Information
***These studies investigates what happens (or happened) as a result of an exposure***
Harm/Risk/Etiology: Importance
- 3 Questions to Ask
How strong is the association between exposure and outcome?
How precise is the estimate of risk?
- If cohort study (Prospective), must use RR (Relative Risk) i.e. Hazard Ratio (HR)
- if case-control study (Retrospective), use OR (Odds Ratio)
Statistical Significance?
-Look for 95% CI around results - if RR or OR cross 1.0 then no effect
Prognosis Studies
- Definitions
- Prognostic Factors
-
Possible outcomes of a disease or condition and
- Frequency with which possible outcomes occur
Synonym is _***natural history***_ of an ailment
-
Prognostic Factors: characteristics or population that can be used to more accurately predict that patient or population’s outcome
- Demographic (e.g. age, gender, etc.)
- Disease-specific (e.g. tumor stage)
- Comorbidity (e.g. diabetes in a cardiac patient)
Which type of study is best for studies of prognosis?
Cohort Studies (prospective)
Also can use Case-control studies
Prognosis Studies: Importance
- Questions to ask
What is the risk of the outcome over time?
How preceise are the estimates of risk?
Prognosis Studies: Validity
- 3 questions to ask
Was there a representative and well-defined sample of subjects? (recruied at common point in illness, inclusion/exclusion criteria, representative setting, similar to your own patients)
Was follow up sufficiently long and complete? (>20% lost to followup, validity is suspect)
Were outcomes measured “blind”?
Prognosis Studies: Referral Bias
- Definition
A systematic error that can occur when patients are selected from different care settings (e.g. tertiary care compared to primary care centers) thus increasing the likelihood of adverse or unfavorable outcomes
Prognosis Studies
- Were objective and unbiased criteria used?
- Outcome of interest is defined before the study starts
- Criteria for diagnosis of outcome of interest should be as objective as possible
- Blinding of individual(s) measuring outcome is best but not alwasy possible
Prognosis Studies
- Reassuring a Concerned Patient/Relative
A valid, precise, and generalizable result of good prognosis is usually very helpful in reassuring
Systematic Review
- Definition
Review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research and to collect and analyze data from studies included
- Research synthesis
- Research integration
- Information synthesis
- Overview
Meta-Analysis
- Definition
Statistical analysis of a collection of analysis results from individual studies for the purpose of integrating the findings
Systematic Reviews: Importance
Odds Ratios
Relative Risk
Systematic Reviews: Validity
- High quality studies means…
- How and what trials were found?
- Are results consistent across studies?
- RCTs
- Should go beyond standard databases, Should include all relevant studies (including worldwide and non-published)
- What criteria used to extract data (think PICO)
- Clinical heterogeneity makes studies less useful
Odds Ratio
- Used
- What does OR of 1 mean?
- Used in case-control studies to copmare whether or not probability of a certain even tis the same in two groups
- The event is equally likely in both groups
Relative Risk
- Used
- What does a RR of 1 mean?
- Used in RCT (randomized clinical trials)or incohort studies to state risk of an event (or developing a disease) relative to exposure
- Means that the treatment did not do anything, as it is equally probable in both groups
Systematic Reviews: Integrating the Findings
- Single Studies
- Studies can disagree with respect to:
- Single Study rarely provides definitive conclusions; reviewing a group of similar studies adds power
- Can disagree with respect to:
- Clinical Significance: the magnitude of effect
- Statistical Significance: the likelihood that an observed effect occurred by chance alone
What does the PICO mnemonic help with?
Asking Answerable Questions
What does PICO stand for?
Patient or population
Intervention
Comparison intervention
Outcome of interest
What is the basis for evidence based medicine?
Asking truly answerable questions
What is the best design for a study of Therapy?
Double blind, randomized control trial
Studies on prognosis seek information on which of the following?
- The reason a particular condition has happened
- The results of a particular intervention
- The outcome of a test for a condition
- The possible outcomes of a disease or condition
- Which patients survive longest
The possible outcomes of a disease or condition