Evidence Synthesis and Systematic Reviews Flashcards
a ____ review Identifies, appraises, and synthesizes every study related to a focused question
a systematic
a ___ review assesses the size and the scope of the literature to identify gaps and research needs
a scoping review
a ___ review attempts to situate evidence in a specific context
a realist review
a ___ synthesis is a method for integrating or comparing findings from qualitative studies
a qualitative synthesis
a ____ ____ analysis is similar to a meta-analysis, but for multiple treatments for the same condition
network meta-analysis
A systematic review attempts to collate all the __ __ that fits _-__ ___ criteria in order to answer a specific ___ ___. It uses __, ___ methods that are selected with a view to __ ___, thus providing more reliable findings from which __ can be drawn and __ made
A systematic review attempts to collate all the empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made
A statistical technique for combining the findings from
independent studies
Can be performed following a systematic review if selected
studies are sufficiently homogenous
Treats the data from different studies as if they were from one large study, rather than simply counting the studies
meta-analysis
5 issues with review articles
- authors of summary reviews are experts in the area of the review
- little or no attempt to be systematic in the formulation of the question
- no searching for evidence
- no summarizing the evidence
- information in summary reviews needs o be taken at face value
synopses of the systematic review process
- formulate PICO method criteria
- a priori hypothesis to explain heterogeneity
- conduct search
- screen titles and abstracts
- review full text of possibly eligible studies
- assess risk of bias, abstract data
- generate summary estimates and CIs. look for explanations of hetergeneity. Rate confidence in estimates of effects
the methods section of a systematic review should include
The variability among studies’ results is termed
heterogeneity
statistical heterogeneity
occurs when the difference among study results is greater than chance alone
- Chi-squared test
- degrees of inconsistency (I2)
two ways statistical heterogenity is reported
- Chi-squared test
- degrees of inconsistency (I2)
The authors should hypothesize possible explanation for
heterogeneity (a priori, when they plan the review) and test their hypotheses in a __ __
The authors should hypothesize possible explanation for
heterogeneity (a priori, when they plan the review) and test their
hypotheses in a subgroup analysis
ways data should be presented for clinical application
RR, OR, or HR,
Continuous variables like weighted mean difference
NNT/NNH
Having 2 or more people participate in each decision at each step of the study guards against errors
Systematic reviewers often report a measure of agreement to quantify their level of agreement on study selection and appraisal of the risk of bias (___)
Having 2 or more people participate in each decision at each step of the study guards against errors
Systematic reviewers often report a measure of agreement to quantify their level of agreement on study selection and appraisal of the risk of bias (KAPPA)
Ideally, systematic review authors should address the risk of bias that can diminish confidence in estimates.
When might a confidence rating decerase?
when there is an increased risk of bias, an inconcsistency, imprecision, indirectness or concern about publication bias (GRADE criteria)
Publication bias
Publication bias is the likelihood that negative results, i.e., studies that do not show a difference or benefit of a treatment, are less likely to be published.
Since publication bias favors publication of research showing a benefit, a meta-
analysis combining on published studies could artificially inflate the real benefit of an
intervention.
how to detect publication bias
- examine whether the smaller studies show bigger effect
- FUNNEL PLOT compares the effect size in different studies with some measure of the variability of the data from each study
- visual assessment
- statistical analysis
interpret this forest plot
• The box represents the relative risk and
its size conveys the relative weight of that
study.
• The horizontal bars represent the
confidence interval for that study.
• The diamond at the bottom is the result
for the combined study results, with the
horizontal points representing the
confidence interval.
Therefor, this interprestation: the horizontal bars crossing the solid vertical lines are not statistically significant–therefore, these studies do not favor one outcome over the other.
- the vertical dashed lines shows the relationship of combined result to the resuls for each study (heterogeneity).
- for this plot, there is littel difference between the combined result any any of the individual studies.
T/F sysnthesis reviews include systematic reviews
ture. synthesis reviews collect all of the available evidence, evaluate its validity, and use it to answer a specific question/
- ideal for answering foreground questions
- incldues systematic reviews.
summary reviews are useful for answering ___ questions
background questions
.Summary reviews: Cover the full breadth of a particular topic, typically providing an overview of the disease etiology, diagnosis,
prognosis, and management
• Useful for answering background questions
• Usually written by experts
Meta analysis: a statistical technique for combining the findings from independent studies
- Can be performed following a systematic review if selected studies are sufficiently __
- Treats the data from different studies as if they were from one large study, rather than simply counting the studies
- Pooling of studies increases __ (i.e., narrows the __ __ [CIs])
- __ __ __ estimate generated facilitates clinical decision making
Meta analysis: a statistical technique for combining the findings from independent studies
- Can be performed following a systematic review if selected studies are sufficiently homogenous
- Treats the data from different studies as if they were from one large study, rather than simply counting the studies
- Pooling of studies increases precision (i.e., narrows the confidence intervals [CIs])
- Single best effect estimate generated facilitates clinical decision making
3 different types of heterogeneity:
- clinical: differences in participants, interventions, or outcomes
- methodological: differences in study design, risk of bias
- statistical: variation in intervention effects or results.
- statistical heterogeneity occurs when the difference among study results is greater than chance alone.
the chi-squared test
assumes the null hypothesis that all the studies are homogeneous, or that each study is measuring an
identical effect, and gives us a p-value to test this hypothesis. If the p-value of the test is low we can reject the hypothesis and heterogeneity is present.
Because the test is often not sensitive enough and the wrong exclusion of heterogeneity happens quickly, a lot of
scientists use a p-value of < 0.1 instead of < 0.05 as the cut-off.
Note: degrees of inconsistency
how to deal with heterogeneity?
• Check your data for mistakes – Go back and see if you maybe typed in something wrong
- Don’t do a meta-analysis if heterogeneity is too high – Not every systematic review needs a meta-analysis
- Explore heterogeneity – This can be done by subgroup analysis or meta-regression
- Perform a random effects meta-analysis – Bear in mind that this approach is for heterogeneity that cannot be explained because it’s due to chance
- Changing the effect measures – Let’s say you use the Risk Difference and have high heterogeneity, then try out Risk Ratio or Odds Ratio
reporting vs publication bias
Reporting bias: failure to report or publish negative or unfavorable results within a study
. Publication bias: failure to publish studies due to the perception that the results are not meaningful (typically occurs with small negative studies).
T/F: Systematic reviews are at risk of presenting misleading results if Reviewers include unpublished studies.
false. . (while they may not have gone through rigorous peer
review, NOT including unpublished studies omits potentially relevant data, and unpublished
studies are the cause of publication bias) you should include unpublished studies in a systematic review.
note:
Evaluating Hetergoeneity in Study Results:
1. Visual Evaluation of variability/heterogeneity
o How similar are the point estimates?
o To what extent do the confidence intervals overlap?
2. Statistical tests evaluating variability/heterogeneity
o Yes-or-no tests for heterogeneity that generate a P value o I2 test that quantifies the variability explained by between-study differences in result
