Evidence Synthesis and Systematic Reviews

Question 1

a ____ review Identifies, appraises, and synthesizes every study related to a focused question

Answer 1

a systematic

Question 2

a ___ review assesses the size and the scope of the literature to identify gaps and research needs

Answer 2

a scoping review

Question 3

a ___ review attempts to situate evidence in a specific context

Answer 3

a realist review

Question 4

a ___ synthesis is a method for integrating or comparing findings from qualitative studies

Answer 4

a qualitative synthesis

Question 5

a ____ ____ analysis is similar to a meta-analysis, but for multiple treatments for the same condition

Answer 5

network meta-analysis

Question 6

A systematic review attempts to collate all the __ __ that fits _-__ ___ criteria in order to answer a specific ___ ___. It uses __, ___ methods that are selected with a view to __ ___, thus providing more reliable findings from which __ can be drawn and __ made

Answer 6

A systematic review attempts to collate all the empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made

Question 7

 A statistical technique for combining the findings from
independent studies

 Can be performed following a systematic review if selected
studies are sufficiently homogenous

 Treats the data from different studies as if they were from one large study, rather than simply counting the studies

Answer 7

meta-analysis

Question 8

5 issues with review articles

Answer 8

1. authors of summary reviews are experts in the area of the review
2. little or no attempt to be systematic in the formulation of the question
3. no searching for evidence
4. no summarizing the evidence
5. information in summary reviews needs o be taken at face value

Question 9

synopses of the systematic review process

Answer 9

1. formulate PICO method criteria
2. a priori hypothesis to explain heterogeneity
3. conduct search
4. screen titles and abstracts
5. review full text of possibly eligible studies
6. assess risk of bias, abstract data
7. generate summary estimates and CIs. look for explanations of hetergeneity. Rate confidence in estimates of effects

Question 10

the methods section of a systematic review should include

Answer 10

Question 11

The variability among studies’ results is termed

Answer 11

heterogeneity

Question 12

statistical heterogeneity

Answer 12

occurs when the difference among study results is greater than chance alone

• Chi-squared test
• degrees of inconsistency (I2)

Question 13

two ways statistical heterogenity is reported

Answer 13

• Chi-squared test
• degrees of inconsistency (I2)

Question 14

The authors should hypothesize possible explanation for
heterogeneity (a priori, when they plan the review) and test their hypotheses in a __ __

Answer 14

The authors should hypothesize possible explanation for
heterogeneity (a priori, when they plan the review) and test their
hypotheses in a subgroup analysis

Question 15

ways data should be presented for clinical application

Answer 15

RR, OR, or HR,

Continuous variables like weighted mean difference

NNT/NNH

Question 16

Having 2 or more people participate in each decision at each step of the study guards against errors

 Systematic reviewers often report a measure of agreement to quantify their level of agreement on study selection and appraisal of the risk of bias (___)

Answer 16

Having 2 or more people participate in each decision at each step of the study guards against errors

 Systematic reviewers often report a measure of agreement to quantify their level of agreement on study selection and appraisal of the risk of bias (KAPPA)

Question 17

 Ideally, systematic review authors should address the risk of bias that can diminish confidence in estimates.

When might a confidence rating decerase?

Answer 17

when there is an increased risk of bias, an inconcsistency, imprecision, indirectness or concern about publication bias (GRADE criteria)

Question 18

Publication bias

Answer 18

Publication bias is the likelihood that negative results, i.e., studies that do not show a difference or benefit of a treatment, are less likely to be published.

Since publication bias favors publication of research showing a benefit, a meta-
analysis combining on published studies could artificially inflate the real benefit of an
intervention.

Question 19

how to detect publication bias

Answer 19

• examine whether the smaller studies show bigger effect
• FUNNEL PLOT compares the effect size in different studies with some measure of the variability of the data from each study
• visual assessment
• statistical analysis

Question 20

interpret this forest plot

Answer 20

• The box represents the relative risk and
its size conveys the relative weight of that
study.
• The horizontal bars represent the
confidence interval for that study.
• The diamond at the bottom is the result
for the combined study results, with the
horizontal points representing the
confidence interval.

Therefor, this interprestation: the horizontal bars crossing the solid vertical lines are not statistically significant–therefore, these studies do not favor one outcome over the other.

• the vertical dashed lines shows the relationship of combined result to the resuls for each study (heterogeneity).
• for this plot, there is littel difference between the combined result any any of the individual studies.

Question 21

T/F sysnthesis reviews include systematic reviews

Answer 21

ture. synthesis reviews collect all of the available evidence, evaluate its validity, and use it to answer a specific question/
- ideal for answering foreground questions
- incldues systematic reviews.

Question 22

summary reviews are useful for answering ___ questions

Answer 22

background questions

.Summary reviews: Cover the full breadth of a particular topic, typically providing an overview of the disease etiology, diagnosis,
prognosis, and management
• Useful for answering background questions
• Usually written by experts

Question 23

Meta analysis: a statistical technique for combining the findings from independent studies

• Can be performed following a systematic review if selected studies are sufficiently __
• Treats the data from different studies as if they were from one large study, rather than simply counting the studies
• Pooling of studies increases __ (i.e., narrows the __ __ [CIs])
• __ __ __ estimate generated facilitates clinical decision making

Answer 23

Meta analysis: a statistical technique for combining the findings from independent studies

• Can be performed following a systematic review if selected studies are sufficiently homogenous
• Treats the data from different studies as if they were from one large study, rather than simply counting the studies
• Pooling of studies increases precision (i.e., narrows the confidence intervals [CIs])
• Single best effect estimate generated facilitates clinical decision making

Question 24

3 different types of heterogeneity:

Answer 24

1. clinical: differences in participants, interventions, or outcomes
2. methodological: differences in study design, risk of bias
3. statistical: variation in intervention effects or results.
   - statistical heterogeneity occurs when the difference among study results is greater than chance alone.

Question 25

the chi-squared test

Answer 25

assumes the null hypothesis that all the studies are homogeneous, or that each study is measuring an identical effect, and gives us a p-value to test this hypothesis. **If the p-value of the test is low we can reject the hypothesis and heterogeneity is present.** **Because the test is often not sensitive enough and the wrong exclusion of heterogeneity happens quickly, a lot of scientists use a p-value of \< 0.1 instead of \< 0.05 as the cut-off.**

Question 26

Note: degrees of inconsistency

Question 27

how to deal with heterogeneity?

Answer 27

• Check your **data for mistakes** – Go back and see if you maybe typed in something wrong * **Don’t do a meta-analysis** if heterogeneity is too high – Not every systematic review needs a meta-analysis * Explore **heterogeneity** – This can be done by subgroup analysis or meta-regression * Perform a **random effects meta-analysis** – Bear in mind that this approach is for heterogeneity that cannot be explained because it’s due to chance * **Changing** the effect measures – Let’s say you use the Risk Difference and have high heterogeneity, then try out Risk Ratio or Odds Ratio

Question 28

reporting vs publication bias

Answer 28

**Reporting bias**: failure to report or publish negative or unfavorable results within a study **. Publication bias**: failure to publish studies due to the perception that the results are not meaningful (typically occurs with small negative studies).

Question 30

T/F: Systematic reviews are at risk of presenting misleading results if Reviewers include unpublished studies.

Answer 30

false. . (while they may not have gone through rigorous peer review, NOT including unpublished studies omits potentially relevant data, and unpublished studies are the cause of publication bias) you should include unpublished studies in a systematic review.

Question 31

note: Evaluating Hetergoeneity in Study Results: **1. Visual Evaluation** of variability/heterogeneity o How similar are the point estimates? o To what extent do the confidence intervals overlap? **2. Statistical tests evaluating** variability/heterogeneity o Yes-or-no tests for heterogeneity that generate a P value o I2 test that quantifies the variability explained by between-study differences in result