Evidence Based Medicine Flashcards

1
Q

Learning Objectives

A

Distinguish between narrative review articles (non-systematic reviews), systematic reviews, and meta-analysis, and understand issues in using them.

Describe characteristics of a good clinical practice guideline, including a focus on clinical outcomes, use of best available evidence, and involvement of multi-professional groups and consumers in development.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is evidence-based medicine?

A

“Evidence-based medicine is the integration of best research evidence with clinical expertise and patient values.”
- David Sackett

Combine: 
1. Best Research evidence
2. Clinical Expertise
&
3. Patient Values
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is “best research evidence”?

A
  1. Systematic Reviews of RCTs
    i. The strongest evidence and statistics
  2. Randomized controlled trials (RCT)
    i. 2nd best evidence
  3. Controlled observational studies
    i. E.g., cohort, case-control, cross-sectional
    ii. medium evidence
  4. Uncontrolled observational study (case series)
    i. okay evidence
  5. Physiologic and animal studies
    i. weak evidence
  6. Unsystematic clinical observations, expert opinion
    i. weak evidence
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Intravenous streptokinase for acute myocardial infarction

A

a. Blood clots in critical vessels can cause MI
b. Streptokinase and other drugs break up blood clots
c. Between 1959 and 1988, 33 RCTs evaluated intravenous streptokinase versus placebo in patients hospitalized for acute MI
d. Total of 36,974 patients enrolled
* Large systematic review of Randomized control trials= awesome evidence

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Forest Plot looking at these different Randomized Control Trials

A

a. Meta-analysis–> found benefit for streptokinase in preventing MI over many randomized controlled trials

b. Explain Forrest Plot
i. The results of meta-analyses are typically displayed as Forest plots, where each line represents a single trial, including the reference, often the year of publication, and the number of subjects in the intervention and control group.
ii. Next is the point estimate and confidence interval from that trial.
iii. If the horizontal line overlaps 1.0, then the trial did not have statistically significant results. When the point estimate is to the left of the line, the intervention group has a better outcome.
iv. When the point estimate is to the right of the line, the control group has a better outcome. The dark black dot at the bottom of the plot is the combined results of all the trials together.

v. The left side of the Figure shows that the effect of treatment with streptokinase on mortality was favourable in 25 of the 33 trials, but in only six was statistical significance achieved.
* Summary: The overall pooled estimate of treatment effect given at the bottom significantly favoured treatment.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Use of Human Albumin

in Critically Ill Patients

A

a. In critically ill patients, serum albumin concentration inversely related to risk of death
b. Human albumin solutions used for shock, burns, and illnesses associated with hypoproteinemia
c. Human albumin costs 30 times more than saline
d. Is it beneficial? Is it cost-effective?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Human Albumin Study-Large Summary of study

A

a. A systematic review of cohort studies meeting specified criteria estimated that for each 2.5 g/l decrement in serum albumin concentration the risk of death increases by between 24% and 56%.
b. The association persists after adjustment for other known risk factors and pre-existing illness, and some commentators have suggested the possibility of the albumin molecule having a direct protective effect.
c. Partly as a result of the association between serum albumin and mortality, Licensed indications for human albumin solution are the emergency treatment of shock and other conditions in which restoration of blood volume is urgent, the acute management of burns, and clinical situations associated with hypoproteinaemia.

d. Compared with other colloidal solutions and with crystalloid solutions, human albumin solutions are expensive.
i. Volume for volume, human albumin solution is twice as expensive as hydroxyethyl starch and over 30 times more expensive than crystalloid solutions such as sodium chloride or Ringer’s lactate.

e. Because of the high cost and limited availability of human albumin, it is imperative that its use should be restricted to the indications for which it has been shown to be effective.
f. To quantify the effect on mortality of human albumin solution in the management of critically ill patients with hypovolaemia from injury or surgery, burns, and hypoproteinaemia, researchers conducted a systematic review of randomised controlled trials.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What the results showed for albumin studies

A

Summary: Found that albumin treatment actually lead to higher rate of harm, when looking at a large meta-analysis of albumin treatment
i. each RCT by itself looked to have varied results.

a. The line down the middle represents a relative risk or odds ratio of 1.0. In this forest plot, the size of the box indicating the point estimate varies with the size of the trial, with larger trials with smaller variance having larger boxes.
b. Larger trials receive more weight in the quantitative analysis than do small trials.
c. In this example, you can see that nearly all the confidence intervals overlap an odds ratio of 1, indicating that the result is not statistically significant.
d. *When all results are combined, albumin was found to be actually harmful, that is, the placebo controls had a better outcome than the patients given albumin.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Why EBM matters

A

Identification of best available evidence and integration of evidence into practice has the potential to:
i. improve health and well-being

ii. Avoid harms and conserve resources

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Systematic Review

A

a. Summary of the best available evidence to address a focused question

b. A systematic review is a type of literature review that collects and critically analyzes multiple research studies or papers.
i. A review of existing studies is often quicker and cheaper than embarking on a new study. Researchers use methods that are selected before one or more research questions are formulated, and then they aim to find and analyze studies that relate to and answer those questions.
ii. Systematic reviews of randomized controlled trials are key in the practice of evidence-based medicine.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Systematic Reviews versus Traditional Literature Reviews

A

a. Literature reviews, like all research, are subject to selection and information bias

b. Systematic reviews use standard methods designed to reduce bias
i. try to reduce bias

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Sources of systematic error (bias)

in literature reviews

A

How the review author:
-defined the study question

  • looked for studies
  • decided what studies to include
  • decided if the studies were any good
  • decided what comparisons to analyze and what outcomes to report
  • interpreted the results
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Systematic vs Traditional Review

A

Benefits of Systematic Review:
a. Systematic Reviews are scientific investigations in themselves.

b. Well Formulated Question: that is, the specific purpose of the review is identified & stated.
c. Comprehensive and Explicit Data Search: Sources and methods of the citation search are identified, and there is an attempt to identify all relevant literature.
d. An Unbiased selection and Abstraction Process – that is, explicit guidelines are used to determine what material is included in and excluded from the review.
e. Critical Appraisal of data – The material in the review is assessed for methodological validity.

f. Appropriate synthesis of the data – the information is systematically integrated. Data limitations and inconsistencies are described. The information may be integrated and weighted or pooled quantitatively (I.e., through meta-analysis).
i. A summary of pertinent findings is provided.

Traditional Review: Broad, unspecified, variable in all factors

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Could a statin lower my risk of having a heart attack?”

"John the patient"
Professor at CU Boulder
60 years old
Well controlled hypertension (BP 130/80)
Never smoker
No fam hx heart disease
No other medical history
Total cholesterol 210, 
    HDL 55
A

What research to seek out:

  1. Seek out Summary and Guidelines
    i. see if there are guidelines
  2. If no guidelines, then look at Pre-appraised research.
    i. These are systematic reviews that have been brought together
  3. Non-appraised research
    i. these are clinical trials by themself
    ii. can be risky, as seein with the albumin study
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Hierarchy of Evidence

A

a. Is there an evidence-based practice guideline that gives me the information I need to optimize John’s care?*
b. If not, is there a systematic review that answers the question of whether a statin could lower John’s risk of having a heart attack?

Cochrane Library: “Statins for the Primary Prevention of Cardiovascular Disease”
i. Meta-analysis was found from randomized clinical trials

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Critical Appraisal of Systematic Reviews

A
  1. Are the results valid?
    i. do you believe them?
  2. Are the valid results meaningful?
    i. is there an actual benefit
  3. Are the valid, meaningful results relevant to my patient / my practice
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Critical Appraisal of Systematic Reviews

Validity

A

Are the results valid?

Is this a systematic review of the best available evidence?

Clearly focused question
Inclusion criteria specify appropriate study designs

Does it describe a comprehensive and detailed search for relevant trials?

Were the individual studies assessed for validity?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

A. Clearly Focused Question

Important for a Systematic review

A

a. The specific purpose of the review is identified & stated

b. The question specifies:
i. Population studied
- Age, gender, health condition, activity, setting

ii. Intervention given
- Drug, surgical procedure, diagnostic test, program, regulation, engineering change, policy, law

iii. Outcomes considered
Condition, illness, behavior

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Unfocused Question

Example of bad question for Systematic Review

A

What is new in treating breast cancer?

Too vague, no specific intervention, population, or outcome

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Formulating a research question

A

a. Poorly focused questions lead to unclear decisions about what research to include and how to summarize it

b. Well-formulated questions guide:
i. determination of final criteria used to select studies to include in the review

ii. what data should be abstracted from included studies

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Clearly Focused Question

A

How effective are antiepileptics for preventing epilepsy following traumatic head injury?

  1. Population: people with traumatic head injury
  2. Intervention: anti-epileptics
  3. Outcome: epilepsy
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

B. Comprehensive and Explicit Data Search

A

a. Search strategy based on explicit criteria developed from the focused question
b. Attempt to identify all relevant literature
c. Sources and methods of the literature search are identified

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

Define inclusion and exclusion criteria

A

P (Types of Population)
I (Types of Intervention(s))
C (Types of Comparisons, controls)
O (Types of Outcome(s))

S (Types of Study designs)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Types of Study Designs

A

a. Included studies should be the best available evidence, based on the Evidence Hierarchy, to address the review question

b. Appropriate studies for inclusion
i. RCTs for treatment, diagnostic tests
ii. Observational studies for hazardous exposures, prognosis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

Locate and select studies

A

a. Comprehensive search strategies to find all relevant studies, to avoid bias

b. Bias = systematic errors that cause results to be consistently distorted in one direction because of nonrandom factors
i. Bias is a Distortion of the results

c. Two main types in systematic reviews:
- Publication bias
- Location bias

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

Looking at studies and what gets published

A

A= All Studies Conducted
+, - and null findings, statistically significant and non-significant results

B=All Studies Published
+, statistically significant

C=all studies that are identified in a systematic review – location bias

*Location Bias—> looking for studies that have a positive result, will have a location bias

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

Location Bias

A

Location Bias: Bias in

a. The publication of research findings in journals with different ease of access or levels of indexing in standard databases, depending on the nature and direction of results.
b. There is also evidence that, compared to negative or null results, statistically significant results are on average published in journals with greater impact factors, and that publication in the mainstream (non grey) literature is associated with an overall greater treatment effect compared to the grey literature

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
28
Q

Bias in Systematic Reviews

A

a. Publication Bias: Tendency for published studies to differ systematically in their results from unpublished studies
i. +, significant studies more likely to be published

b. Location Bias: Tendency for high profile, widely disseminated studies to differ systematically in their results from low profile, less widely disseminated studies
i. +, significant studies easier to find

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
29
Q

How to reduce Publication and Location Bias in Systematic Reviews

A

a. Avoid language restrictions
b. Search more than one electronic database (hundreds exist)
c. Search other types of documents (abstracts, reports, letters, dissertations)
d. Check references of other studies, books
e. Contact experts or organizations
f. Contact trialists for unpublished outcomes

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
30
Q

C. Unbiased Selection and Abstraction Process

A

a. Explicit guidelines used to determine what studies to include and exclude
i. Participants, Interventions, Comparisons, Outcomes, Study designs

b. Screen identified records for eligibility based on stated inclusion criteria
i. Best practice: two investigators independently screen and assess relevance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
31
Q

D: Critical appraisal of identified studies

A

Quality assessment of each included study using standardized criteria

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
32
Q

Internal Validity

A

a. Extent to which each study’s results are valid for circumstances being studied

b. Beneficial effects of trials are likely to be overestimated if:
i. Inadequate allocation concealment
ii. Lack of blinding
iii. Inadequate randomization generation
iv. High dropout rates

(Concealment of group allocation means it is not possible to tell to which group the next enrolled subject will be assigned, and therefore characteristics of the subject cannot influence group assignment.)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
33
Q

Quality Assessment: How?

A

a. Inclusion criteria and screening
i. E.g., only include double-blind RCTs with >80% follow-up

b. Component assessment of included trials
i. Allocation concealment
ii. Randomization sequence generation
iii. Blinding
iv. Loss to follow-up (dropouts)

c. Scales for quality assessment

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
34
Q

Q1: Is this a systematic review of best available evidence?

A

Look for:

a. A clearly focused question
i. population, intervention, outcomes

b. Inclusion of the appropriate sort of studies to address the review question
i. RCTs for treatment, diagnostic tests
ii. Observational studies for hazardous exposures, prognosis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
35
Q

Statins for the Primary Prevention of Cardiovascular Disease”

Critiquing the study

A

P: People without CVD
I: Statins
O: CVD

Types of studies: RCTs comparing treatment with statins for at least 12 months with placebo or usual care; follow-up at least six months

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
36
Q

Question #2: Does the review describe a comprehensive and detailed search for relevant studies?

A

a. Look for:
- Which bibliographic databases were used
- Follow up from reference lists
- Personal contact with experts
- Searches for unpublished studies
- Search for non-English language studies

b. Consider:
- Explicit use of inclusion and exclusion criteria
- Use of two independent reviewers
- Standardized abstraction of study data

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
37
Q

Statins for the Primary Prevention of Cardiovascular Disease”

a comprehensive and detailed search for relevant studies

A

a. Searched three bibliographic databases
b. Reviewed reference lists
c. Looked for unpublished studies
d. No language restrictions
e. Two authors independently reviewed search results
f. Eligibility based on inclusion criteria
g. Standard form to abstract data

38
Q

Question #3: Were the individual studies assessed for validity?

A

a. Look for whether and how the review authors considered the validity of the studies they have identified.
b. Lack of validity may affect the studies’ results and therefore the review’s results.

39
Q

Statins for the primary prevention of cardiovascular disease

A

a. For validity, look for possible risk of bias
b. Important to see if there is any bias that will cloud reporting of data
c. Example is the bias with possible pharm sponsorship

40
Q

Trials with pharmaceutical sponsorship

A

a. More likely to report results and conclusions that favor drug over placebo, due to biased reporting and/or interpretation of trial results
b. Reporting of adverse events generally poor, with failure to provide details of severity and type of adverse events or to report on health-related quality of life.

41
Q

Critical Appraisal of Systematic Reviews

If the results are meaningful

A

Are the valid results meaningful?

  • Are the results consistent across studies?
  • What is the size of the treatment effect?
  • How precise is the treatment effect?
42
Q

E: Appropriate Synthesis of Data

A

a. Information is systematically integrated and described

b. A summary of all pertinent findings is provided

43
Q

Narrative Synthesis of Data

A

a. Structured summary and discussion of individual study characteristics and effect estimates
b. *All relevant results should be presented

c. Use structured approach to presentation:
i. identify similarities or differences between studies
ii. Look for patterns of effects
iii. Organize into clusters or groupings by intervention, population, outcome, etc

44
Q

Ways to organize narrative synthesis of data

A
  1. Cochrane Summary of Findings Table

2. Presenting Results with Harvest Plot

45
Q

Synthesis of Data

Meta-analysis

A

Quantitative (“meta-analysis”)
a. Statistical method for combining effect estimates of multiple studies to produce a single common estimate of effect

b. Improves precision by combining all available data

46
Q

Meta-analysis

A

a. Optional part of a systematic review

b. This is a statistical method for analyzing results, may not be part of systematic review

47
Q

Averaging vs. Weighting Studies

A

a. A simple average gives each study equal weight
b. This seems intuitively wrong: some studies are more likely to give an answer closer to the ‘true’ effect than others

c. Meta-analysis gives more weight to studies which give more information
i. More participants
ii. More events
iii. Lower variance

48
Q

Wiki Meta Analysis

A

A meta-analysis is a statistical analysis that combines the results of multiple scientific studies.

The basic tenet behind meta-analyses is that there is a common truth behind all conceptually similar scientific studies, but which has been measured with a certain error within individual studies. The aim then is to use approaches from statistics to derive a pooled estimate closest to the unknown common truth based on how this error is perceived. In essence, all existing methods yield a weighted average from the results of the individual studies and what differs is the manner in which these weights are allocated and also the manner in which the uncertainty is computed around the point estimate thus generated. In addition to providing an estimate of the unknown common truth, meta-analysis has the capacity to contrast results from different studies and identify patterns among study results, sources of disagreement among those results, or other interesting relationships that may come to light in the context of multiple studies.[1] Meta-analysis can be thought of as “conducting research about previous research.” Meta-analysis can only proceed if we are able to identify a common statistical measure that is shared among studies, called the effect size, which has a standard error so that we can proceed with computing a weighted average of that common measure. Such weighting usually takes into consideration the sample sizes of the individual studies, although it can also include other factors, such as study quality.

A key benefit of this approach is the aggregation of information leading to a higher statistical power and more robust point estimate than is possible from the measure derived from any individual study.

49
Q

When can we do a meta-analysis?

A

a. More than one study has estimated an effect on the outcome
b. The outcome has been measured in similar ways
c. There are no differences in the studies’ characteristics that are likely to substantially affect the outcome

d. The data are available
i. take care with interpretation when only some data are available

50
Q

What if the results of the included studies differ from each other?

(Determining if we can do meta-analysis)

A

a. Heterogeneity: variation between the studies’ results
b. Types of Heterogeneity
1) “Clinical” – differences in how the studies were conducted lead to expected differences in results
2) “Statistical” – statistical tests show study results differ more than expected by chance

51
Q

Causes of heterogeneity

A

Differences between studies with respect to:

  1. Patients: diagnosis, in- and exclusion criteria, etc.
  2. Interventions: type, dose, duration, etc.
  3. Outcomes: type, scale, cut-off points, duration of follow-up, etc.
  4. Quality and methodology: randomised or not, allocation concealment, blinding, etc.
52
Q

How to deal with statistical heterogeneity

A
  1. Do not pool at all
  2. Ignore heterogeneity: use fixed effect model
  3. Allow for heterogeneity: use random effects model
  4. Explore heterogeneity through special statistical methods
53
Q

Meta-Analysis: Advantages

A

a. Summary statistic: translates results from different studies to common measure of effect
b. Reliability: provides more accurate estimate of effect size
c. Power: overcomes problem of low statistical power in studies with small sample sizes

54
Q

Meta-Analysis: Disadvantages

A

a. May inappropriately combine heterogeneous studies (“adding apples and oranges”)
b. Does not control for bias (“garbage in, garbage out”)
c. Problems in interpretation: what does the summary statistic mean?

55
Q

Q#4: Are the results consistent across studies?

A

Consider whether:
a. The results of all the included studies are clearly displayed / described

b. The results were similar from study to study
i. Qualitatively (same direction)
ii. Statistically

c. The reasons for any variations are discussed

56
Q

Question #5: What is the size of the treatment effect?

A

Consider:
-If you are clear about the review’s ‘bottom line’ results

  • What these results are (numerically if appropriate)
  • How were the results expressed (NNT, odds ratio, etc.)
  • Is the size of the effect clinically important?
57
Q

Question #6: How precise is the estimate of effect?

A

a. Are the results presented with confidence intervals?

b. Do the confidence intervals demonstrate statistical significance?

58
Q

Critical Appraisal of Systematic Reviews

Are the valid, meaningful results relevant to my patient / practice?

A

Are the valid, meaningful results relevant to my patient / practice?
i. Can the results be applied to my patient / practice?

ii. Were all important outcomes considered?
iii. Are the benefits worth the harms and costs?

59
Q

Question #7: Can the results be applied to my patient / practice?

A

Consider whether:
a. The patients covered by the review could be sufficiently different from your practice population to cause concern about applying them

b. Your practice setting is likely to differ much from that of the review
c. The treatment is feasible in your practice

60
Q

Question #8: Were all important outcomes considered?

A

a. Were all relevant outcomes examined in the review?

b. Were harms examined in the review?

61
Q

Question #9: Are benefits worth the harms and costs?

A

Consider:
a. Even if this question is not addressed by the review, what do you think?

b. What are the patient’s values and expectations in terms of benefits and harms?

62
Q

Interpretation - Weighing up benefit and harm

A

A treatment might cure acne instantly, but kill one person in 10,000 (very important as acne is not life threatening).

63
Q

Statins for the Primary Prevention of Cardiovascular Disease”

Applicability and Outcome

A

a. Applicability: Studies included adults 18+ without prior CVD, regardless of lipid levels, in high income countries
b. Feasible in my practice: already treat high-risk patients with statins
c. Outcomes: all-cause mortality; fatal & non-fatal CVD, CHD & stroke; revascularization (CABG, angioplasty) rates; serious adverse events

64
Q

“Statins for the Primary Prevention of Cardiovascular Disease”:
Are benefits worth the harms and costs?

A

a. Benefits: reduced death, CVD, CHD, stroke, revascularization
b. Harms: No evidence risk of serious adverse events

c. But:
i. For harms - power low, only one year f/u, few studies of less serious but potentially important AE
ii. All but one trial fully or partially funded by pharmaceutical industry

d. Is our information on the risks and benefits accurate and unbiased?

65
Q

Remaining issues

A

a. Feasibility, desirability and cost-effectiveness of routinely treating adults 50+ with statins
b. Patients’ views on life-long drug therapy
c. Limited evidence on less serious adverse events and QOL
d. Use of alternative public health strategies to reduce cholesterol

66
Q

IOM (NAM) Standards

A
  1. Establish transparency (process and funding explicit and public)
  2. Manage conflicts of interest (disclosure, divestment, exclusion)
  3. Group composition (multidisciplinary-methodology experts, clinicians, patients/consumers)
  4. Collaboration/coordination with systematic evidence review
  5. Establish and provide explanation for, and strength of recommendation
    - Describe benefits and harms
    - Summarize evidence and evidence gaps (quality, quantity, consistency)
    - Describe any influence of other factors (values, opinions, theory, clinical experience)
    - Provide level of confidence/certainty, rating of strength, and explanation of difference in opinion
  6. Provide clear, standardized articulation of recommendation
  7. External review by full spectrum of relevant stakeholders (experts, organizations, agencies, patients, public representatives)
  8. Provide updating with monitoring of the literature and updates based on new information
67
Q

Appraisal of Guidelines for Research and Evaluation (AGREE)

A

a. Canadian Institute of Health Research, AGREE Consortium

b. AGREE instrument assesses methodological rigor and transparency with which a guideline is developed to:
- assess the quality of guidelines
- provide a methodological strategy for the development of guidelines
- inform what information and how information ought to be reported in guidelines

68
Q

Comparison of guideline groups

A

Multi-issue guideline groups
GRADE: Grading of recommendations assessment, development and evaluation
USPSTF: U.S. Preventive Services Task Force (US)
SORT: Strength of Recommendation Taxonomy
NICE: National Institute for Health and Clinical Excellence (UK)
OCEMB: Oxford Centre for Evidence-based Medicine (UK)
SIGN: Scottish Intercollegiate Guidelines Network (Scotland)
NHMRC: National Health and Medical Research Council (AU)
ICSI: Institute for Clinical Systems Improvement (MN)

Special issue guideline groups
EGAPP: Evaluation of Genomic Applications in Practice and Prevention (US)
AHAPG: American Heart Association practice guidelines (US)
ESC: European Society of Cardiology (EU)
EULAR: European League Against Rheumatism (EU)

69
Q

The U.S. Preventive Services Task Force (USPSTF)

A

a. Independent panel of nationally recognized, non-federal experts experienced in primary care, prevention, evidence-based medicine, and research methods

b. Charged by the US Congress to:
i. review the scientific evidence for clinical preventive services and
ii. develop evidence-based recommendations for the health care community

c. Recognized in the Affordable Care Act

70
Q

Steps in explicit process

A

a. Define the question about the provision of a preventive service within an analytic framework
b. Define and retrieve relevant evidence
c. Judge the quality of individual studies and the adequacy of evidence for key questions
d. Synthesize and judge the adequacy of the body of evidence across key questions
e. Judge the certainty of net benefit (balance of benefits and harms)
f. Link magnitude and certainty of net benefit to a recommendation statement/letter grade

71
Q
  1. Management of conflict of interest
A

Strict COI process—reported on website

a. Pre-meeting assessment and review of conflicts:
i. Financial
ii. Professional

b. Assignment of conflict rating that limits participation based on level of conflict
- No/minimal conflict=total participation
- Small (professional) conflict=participate but can’t lead
- Strong conflict=must recuse/leave the room

c. No “divestment” requirement

72
Q
  1. Collaboration/coordination between systematic review and guideline development
A

a. Two leads assigned to every topic
b. Leads and TF staff have direct input into and decision-making in the work plan, analytic framework, key questions, and draft and final systematic review report

73
Q

Retrieve relevant evidence and judge study quality

A

a. Good quality: high internal validity, outcomes can be confidently assigned to the factors under study (refer to Cochrane or Evidence-based Practice Center methodology for criteria)
b. Fair quality: there are some minor threats to internal validity
c. Poor quality: there are major threats to internal validity such that competing hypothesis regarding study outcomes cannot be confidently excluded

74
Q

Evaluate the body of evidence for each key question

A

Critical appraisal questions:
1. Do the studies have the appropriate research design to answer the key question?

  1. To what extent are the studies of high quality (internal validity)?
  2. To what extent are the studies generalizable to the US population (external validity)?
  3. How many studies and how large have been done to answer the key question (precision of the evidence)?
  4. How consistent are the studies?
  5. Are there additional factors supporting conclusions?
75
Q

Judge the body of evidence for each key question

A

a. Convincing evidence: derived from several high-quality studies with consistent, logical results generalizable to the US primary care population and setting
b. Adequate evidence: most but not all of the 6 questions are answered satisfactorily
c. Inadequate evidence: conflicting or poor quality studies individually or in aggregate

76
Q

Judge the certainty of net benefit

A

a. Based on the evidence, estimate the magnitude of benefit or potential benefit and harm or potential harm
b. Estimate net benefit (benefits minus harms)
c. Base judgment of certainty of net benefit through applying critical appraisal questions to the entire body of evidence, or across the chain of evidence

77
Q

Certainty

A

a. High: evidence include consistent results from good studies in primary care populations assessing effects on health outcomes; conclusion is unlikely to be affected by future studies
b. Moderate: evidence is sufficient to determine effects on health outcomes but is constrained by issues raised in critical appraisal; additional information from future studies could change the conclusion
c. Low certainty: evidence is insufficient to assess effects on health outcomes; additional information from future studies may allow for assessment

78
Q

Potential benefits of screening

A

Early detection and treatment improves morbidity and/or mortality (important health outcomes)

79
Q

Potential harms of screening

A

a. There are 5 things that can happen with a screening test, and 4 of them are bad:
1) False negative test (false reassurance, delay in diagnosis of treatable condition)

2) False positive test (unnecessary and potentially harmful diagnostic tests, treatment, and labeling)
3) Over-diagnosis (true positive, but disease wouldn’t progress and treatment unnecessary)
4) No benefit from early detection (diversion of resources from effective services)
b. Also, there may be harms intrinsic to the test itself

80
Q

Harms of screening

A
  1. Radiation
  2. Pain
  3. Anxiety, distress, and other psychological responses
  4. False positives, false negatives, additional imaging and biopsies
  5. Overdiagnosis: diagnosis and treatment of a cancer that would never have progressed to become clinically detectable in the patient’s lifetime
81
Q

Modeling

A

a. Base case—no screening:
i. Median expected lifetime risk of mortality from breast cancer for a women alive at age 40 who undergo no mammography is 3% (for every 1,000 women alive at age 40 the ultimate cause of death for 30 would be breast cancer)

b. Screening age 50-69:
i. Reduces the number of deaths in 1,000 women by about 7 (from 30 to 23)

82
Q

Summary: benefits

Screening

A

a. For every 1000 women alive at age 40, we can expect 30 will ultimately die from breast cancer in the absence of screening
b. We can reduce that number by about 7 by screening women with mammography every other year from age 50 to age 74
c. We can reduce that number by an additional 1 by starting screening at age 40

83
Q

Incremental benefits and harms of extending screening from age 50-69 to 40-69 (1,000 women)

A

a. Benefits:
May increase the deaths averted from 7 to 8

b. Harms:
i. Total number of mammograms increases from 8,944 to 13,865
ii. Total number of false positive results increases from 780 to 1,250
iii. Total number of unnecessary biopsies increases from 55 to 88

c. The rate of false positives prompting unneeded intervention is much greater for younger women: half of all women in their 40’s can be expected to require some additional testing

84
Q

Age 40-49

Breast Screening

A

a. Incremental benefit of starting at age 40 vs age 50 is small
b. Harms, while not life threatening, are extraordinarily common, and are substantially increased by starting at age 40 rather than 50
c. Net benefit is small

85
Q
  1. Articulation of recommendation
A

Standardized “Recommendation and Rationale Statement”

  • Summary of recommendations and evidence
  • Structured Rationale (summary of the TF’s reasoning)
  • Clinical Considerations (how to provide/offer service)
  • Other considerations (cost/cost effectiveness, evidence gaps)
  • Discussion (details of evidence)
  • Recommendations of others
  • References and tables
  • One page clinical summary (for clinicians)
86
Q
  1. External review
A

a. In addition to stakeholder input already described, content experts are identified for peer-review and recommendation statements are edited as appropriate
b. Journal provides additional (though expedited) external review

87
Q

Amended 2009 recommendation

A

The decision to start regular, biennial screening mammography before the age of 50 years should be an individual one and take into account patient context, including the patient’s values regarding specific benefits and harms. (Grade C recommendation)

88
Q
  1. Update
A

a. Review every recommendation every 5 years

b. Ongoing reactive literature scan for relevant new studies that may require a sooner update

89
Q

Pros of process

A

a. Creates recommendations that are based on evidence through a transparent, explicit process that protects against the influence of politics, economics and advocacy
b. Provides primary care clinicians with standardized set of recommendations that, if applied in their practices, can confidently be expected to improve the health of patients
c. Provides other audiences with information that can inform decisions about health care delivery systems, using a high bar of evidence to make recommendations

90
Q

Cons of process

A

a. The most commonly use letter grade is the I: what should clinicians do in the face of insufficient evidence?
b. No systematic consideration of patient values, or cost and cost-effectiveness
c. The process is expensive and time consuming
d. Research often doesn’t answer vital questions, such as periodicity, age to start or stop screening, or effectiveness in diverse populations
e. Uncertain how well the process works in the setting of rare diseases