Systemic Reviews and Meta-Analyses

Question 1

What type of study design has the highest level of evidence?

Answer 1

Systematic Reviews

Question 2

What is a systematic review?

Answer 2

A review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research

Question 3

What is a meta-analysis?

Answer 3

The use of statistical techniques (usually in a systematic review) to integrate the results of independent studies

Question 4

What are the two steps of a meta-analysis?

Answer 4

1. Extract the data and main result from each individual study
2. Calculate a pooled average result (greater weight given to studies that provide more information)

Question 5

What is the difference between systematic reviews and meta-analyses?

Answer 5

A systematic review is a structured summary of evidence from multiple studies without performing quantitative analysis, while a meta-analysis goes a step further by statistically combining the data from multiple studies to provide a single, quantitative estimate of the effect size.

Question 6

What are some issues with combining all study results into one overall result?

Answer 6

Clinical reasons
- Do all the studies address the same question?
a. Similar populations?
b. Similar interventions and/or comparisons
c. Similar outcomes

Statistical reasons
- Study heterogeneity
a. Are the results between studies consistent?
b. Q or I^2 statistics

Question 7

What is the purpose of systematic reviews?

Answer 7

Thousands of papers are published every day, it is impossible to read them all. So systematic reviews compile all of the current work in the field

Question 8

What is the Cochrane Collaboration?

Answer 8

They are an international organization of 30,000 volunteer researchers in 53 review groups

Internationally recognzed as the highest standard in evidence-based healthcare (Due to specific methodology and no industry funding for research)

Question 9

What are the components of systematic reviews?

Answer 9

1. Question of interest (study protocol, RCT only or include observational studies)
2. Comprehensive literature search (hundreds to thousands of search terms, performed by librarians, search in multiple journal databases)
3. Identify appropriate studies for inclusion (read through abstracts)
4. Extract data
5. Analyze data (if doing a meta-analysis)
6. Interpret results (write review)

Question 10

What are some factors that can hurt the quality of a systematic review?

Answer 10

1. Lack of transparency or incomplete reporting
2. Methods for identifying studies (not comprehensive in search)
3. Inclusion/exclusion criteria
4. Quality assessment (critical assessment)

Question 11

What are some factors that can hurt the quality of a meta-analyses?

Answer 11

1. Problems if done outside of a systematic review
2. Inappropriate inclusion/exclusion criteria
3. Garbage in = garbage out (including results from poor quality studies)
4. No measure or consideration of heterogeneity
5. Potential problems if publication bias (most published findings are successful in supporting their hypothesis, overestimate benefit)

Question 12

How can the quality of a systematic review be evaluated?

Answer 12

1. Identify PICO statement
2. Did the authors look for appropriate papers
   a. Do inclusion/exclusion criteria make sense
   b. Do we know how and where they searched for papers
   c. Were all important, relevant studies included (examine bibliography, contact authors)
3. Did the authors assess the quality of the included studies (at least two authors should be involved in a validated quality assessment)
4. If there was a meta-analysis was done, was it appropriate to do so?
   a. Were the populations of the studies similar, were the interventions and comparisons similar between studies
   b. Was a measure of heterogeneicity reported
5. What is the overall result of the review?
6. Do these results apply to my patients
   a. Are the benefits worth the risk
7. Who paid for the study (potential conflict of interest)