Unit 2 - Getting Useful Information Flashcards
why do we grade evidence?
- to know if we have confidence in an individual study (if this is the best data available)
- when we are putting together evidence from a group of studies, we need to know which ones matter more
which study times yield more conclusive results for interventions?
RCT and systematic reviews of RCTs
which study times yield more conclusive results for diagnostic tests?
cohort (X-sectional studies)
what is the usefulness equation?
usefulness = (relevance x validity) / work
what is the American College of Cardiology?
a popular level of evidence system that weighs “general agreement” equal to research evidence
-argue they need to make decisions in absence of definitive evidence
what are ACC classes?
I - evidence and/or general agreement; useful and effective
II - conflicting evidence/divergence of opinion
III - evidence/general agreement; NOT useful/effective, may be harmful
what is the American College of Obstetrics and Gynecology evidence?
early adopter of grading systems for technical bulletins (guidelines)
what are ACOG levels of evidence?
I: at least one properly designed RCT
II-1: controlled trials without randomization
II-2: cohort/case-control studies (>1 site)
II-3: time series w/ or w/o intervention and dramatic results in uncontrolled experiments
III: opinions of respected authorities or reports of expert committees
what are ACOG levels of recommendations?
A: recommendations are based on good and consistent scientific evidence
B: recommendations are based on limited or inconsistent scientific evidence
C: recommendations are based primarily on consensus and expert opinion
what is the CEMB?
center of evidence-based medicine
- one of the first systems; popular and very detailed
- categories for each type of questions
what is PC_SORT?
primary care - strength of recommendation taxonomy
- consensus discussion to arrive at approach
- attempts to simplify
- values POEMs over DOEs
- strength of recommendations - strength and consistency
what is US preventive services task force?
one of the first guideline organizations to use grading
- has evolved over time
- balance between:
- -certainty of evidence (quality and sample size of accumulated evidence)
- -magnitude of benefit (benefit to harm comparison favors screening)
what is GRADE?
grading of recommendations assessment, development, and evaluation
- rapidly becoming standard system for large guideline organizations
- ongoing assessment and evaluation
what are USPSTF grades of recommendations?
A - strong recommendation = substantial evidence, benefits outweigh harms (high net benefit)
B - recommends = substantial/moderate evidence, benefits outweigh harms (high to moderate benefit)
C - no recommendation = high/moderate evidence, but benefits/harms too close (small net benefit)
D - recommends against = high/moderate evidence, harms outweigh benefits (zero/negative net benefit)
I - evidence is insufficient to recommend
what does it mean if you give a grade of “A-D+I”?
A/B: offer/provide this service
C: offer/provide this service only if other considerations support offering or providing service in an individual patient
D: discourage use of service
I: if service is offered, patients should understand uncertainty about benefits/harms (read clinical considerations section)
what are advantages of GRADE?
- developed by international guideline developers
- clear separation between quality of evidence and strength of recommendations
- explicit comprehensive criteria for up/downgrading quality of evidence ratings
- explicit evaluation of importance of outcomes of alternative management strategies
- transparent process of moving from evidence to recommendations
- explicit acknowledgement of values and preferences
- clear, pragmatic interpretation of strong VS weak recommendations for clinicians, patients, and policy makers
- useful for systematic reviews and health technology assessments as well as guidelines
what are the 4 qualities of GRADE evidence?
high: further research is unlikely to change confidence in estimate of effect
moderate: further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate
low: further research is very likely to have an important impact on our confidence in the estimate of effect, and is likely to change the estimate
very low: any estimate of effect is uncertain
what quality of evidence are randomized trials? observational studies? how are they up/downgraded?
randomized: high
observational: low
up/downgraded depending on strengths and limitations
what are factors that might decrease quality of evidence?
- study validity limitations
- inconsistency of results between studies
- indirectness of evidence (wrong outcomes, populations)
- imprecision of estimate of effect (P value, CI)
- publication bias
what are factors that might increase quality of evidence?
- large magnitude of effect (OR, RR)
- plausible confounding which would have reduced demonstrated effect (but there was still an effect seen, meaning effect must be great)
- dose-dependent gradient
don’t change validity
how can there be uncertainty in strengths of recommendation?
- balance between desirable and undesirable effects
- variability in values/preferences
- whether intervention represents a wise use of resources (cost-effectiveness)
how should patients interpret strength of recommendations?
most people would want the recommended course of action, but many would not
how should clinicians interpret strength of recommendations?
different choices will be appropriate for different patients
-help patient arrive at a management decision consistent with values and preferences
how should policy makers interpret strength of recommendations?
policy making will require substantial debate and involvement of many stakeholders
what is the “ideal pre-appraised” source?
- meets usefulness criteria
- selection, focus, structured data, clear recommendations, context, statistics, clear writing - has explicit, rigorous, and easy-to-use system for grading a piece of evidence and/or strength of recommendation
- a reference list isn’t good enough anymore
