Unit 2 - Getting Useful Information

Question 1

why do we grade evidence?

Answer 1

• to know if we have confidence in an individual study (if this is the best data available)
• when we are putting together evidence from a group of studies, we need to know which ones matter more

Question 2

which study times yield more conclusive results for interventions?

Answer 2

RCT and systematic reviews of RCTs

Question 3

which study times yield more conclusive results for diagnostic tests?

Answer 3

cohort (X-sectional studies)

Question 4

what is the usefulness equation?

Answer 4

usefulness = (relevance x validity) / work

Question 5

what is the American College of Cardiology?

Answer 5

a popular level of evidence system that weighs “general agreement” equal to research evidence
-argue they need to make decisions in absence of definitive evidence

Question 6

what are ACC classes?

Answer 6

I - evidence and/or general agreement; useful and effective
II - conflicting evidence/divergence of opinion
III - evidence/general agreement; NOT useful/effective, may be harmful

Question 7

what is the American College of Obstetrics and Gynecology evidence?

Answer 7

early adopter of grading systems for technical bulletins (guidelines)

Question 8

what are ACOG levels of evidence?

Answer 8

I: at least one properly designed RCT
II-1: controlled trials without randomization
II-2: cohort/case-control studies (>1 site)
II-3: time series w/ or w/o intervention and dramatic results in uncontrolled experiments
III: opinions of respected authorities or reports of expert committees

Question 9

what are ACOG levels of recommendations?

Answer 9

A: recommendations are based on good and consistent scientific evidence
B: recommendations are based on limited or inconsistent scientific evidence
C: recommendations are based primarily on consensus and expert opinion

Question 10

what is the CEMB?

Answer 10

center of evidence-based medicine

• one of the first systems; popular and very detailed
• categories for each type of questions

Question 11

what is PC_SORT?

Answer 11

primary care - strength of recommendation taxonomy

• consensus discussion to arrive at approach
• attempts to simplify
• values POEMs over DOEs
• strength of recommendations - strength and consistency

Question 12

what is US preventive services task force?

Answer 12

one of the first guideline organizations to use grading

• has evolved over time
• balance between:
• -certainty of evidence (quality and sample size of accumulated evidence)
• -magnitude of benefit (benefit to harm comparison favors screening)

Question 13

what is GRADE?

Answer 13

grading of recommendations assessment, development, and evaluation

• rapidly becoming standard system for large guideline organizations
• ongoing assessment and evaluation

Question 14

what are USPSTF grades of recommendations?

Answer 14

A - strong recommendation = substantial evidence, benefits outweigh harms (high net benefit)
B - recommends = substantial/moderate evidence, benefits outweigh harms (high to moderate benefit)
C - no recommendation = high/moderate evidence, but benefits/harms too close (small net benefit)
D - recommends against = high/moderate evidence, harms outweigh benefits (zero/negative net benefit)
I - evidence is insufficient to recommend

Question 15

what does it mean if you give a grade of “A-D+I”?

Answer 15

A/B: offer/provide this service
C: offer/provide this service only if other considerations support offering or providing service in an individual patient
D: discourage use of service
I: if service is offered, patients should understand uncertainty about benefits/harms (read clinical considerations section)

Question 16

what are advantages of GRADE?

Answer 16

1. developed by international guideline developers
2. clear separation between quality of evidence and strength of recommendations
3. explicit comprehensive criteria for up/downgrading quality of evidence ratings
4. explicit evaluation of importance of outcomes of alternative management strategies
5. transparent process of moving from evidence to recommendations
6. explicit acknowledgement of values and preferences
7. clear, pragmatic interpretation of strong VS weak recommendations for clinicians, patients, and policy makers
8. useful for systematic reviews and health technology assessments as well as guidelines

Question 17

what are the 4 qualities of GRADE evidence?

Answer 17

high: further research is unlikely to change confidence in estimate of effect
moderate: further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate
low: further research is very likely to have an important impact on our confidence in the estimate of effect, and is likely to change the estimate
very low: any estimate of effect is uncertain

Question 18

what quality of evidence are randomized trials? observational studies? how are they up/downgraded?

Answer 18

randomized: high
observational: low

up/downgraded depending on strengths and limitations

Question 19

what are factors that might decrease quality of evidence?

Answer 19

1. study validity limitations
2. inconsistency of results between studies
3. indirectness of evidence (wrong outcomes, populations)
4. imprecision of estimate of effect (P value, CI)
5. publication bias

Question 20

what are factors that might increase quality of evidence?

Answer 20

1. large magnitude of effect (OR, RR)
2. plausible confounding which would have reduced demonstrated effect (but there was still an effect seen, meaning effect must be great)
3. dose-dependent gradient

don’t change validity

Question 21

how can there be uncertainty in strengths of recommendation?

Answer 21

1. balance between desirable and undesirable effects
2. variability in values/preferences
3. whether intervention represents a wise use of resources (cost-effectiveness)

Question 22

how should patients interpret strength of recommendations?

Answer 22

most people would want the recommended course of action, but many would not

Question 23

how should clinicians interpret strength of recommendations?

Answer 23

different choices will be appropriate for different patients

-help patient arrive at a management decision consistent with values and preferences

Question 24

how should policy makers interpret strength of recommendations?

Answer 24

policy making will require substantial debate and involvement of many stakeholders

Question 25

what is the “ideal pre-appraised” source?

Answer 25

1. meets usefulness criteria
   - selection, focus, structured data, clear recommendations, context, statistics, clear writing
2. has explicit, rigorous, and easy-to-use system for grading a piece of evidence and/or strength of recommendation
   - a reference list isn’t good enough anymore