Systematic reviews and meta-analyses

Question 1

What is the role of systematic reviews and meta analyses in evidence based practice?

Answer 1

They provide a structured summary of research results prepared and executed using well-defined and precise methodologies.

The systematic review is often but not always accompanied by meta-analysis which is a statistical analysis of the combined results of these studies.

Question 2

What is a systematic review?

Answer 2

Review of clearly formulated question which uses relevant research to answer the question.

The review uses systematic and explicit methods to identify, select, and critically appraise relevant research and to collect and analyze data from the studies included.

Question 3

What is the objective of systematic reviews?

Answer 3

Comprehensive and trustworthy means of synthesizing the literature.

Prepared very systematically with explicit, transparent, and predefined methods to limit bias.

Results can be summarized from quantitative, qualtitative or a combination of both studies.

Meta-analysis is not necessary but can provide a statistical summary of the results. If no meta analysis then review is synthesized in words.

Question 4

What can systematic reviews synthesize?

Answer 4

Quantitative studies (meta analysis / narrative synthesis)

Qualitative studies (aggregate or integrate findings from primary qualitative research studies into themes or metaphors)

Quantitative and qualitative studies (mixed methods review)

Question 5

What are the advantages of systematic reviews?

Answer 5

Improve the dissemination of evidence

Hasten assimilation of research into practice

Assist in clarifying the heterogeneity of conflicting results between studies

Establish generalisability of the overall findings

Improve the understanding of a particular phenomenon or situation

Can guide decision making

Can set a research agenda

Question 6

What kinds of systematic reviews should be used for discussing the effectiveness of interventions?

Answer 6

Systematic reviews of RCTs (Most common type)

Question 7

What kind of systematic reviews should be used for questions about diagnosis?

Answer 7

Synthesis about the accuracy of the test or instrument

Question 8

What kind of systematic reviews should be used for questions about prognosis?

Answer 8

Synthesis of cohort studies for questions about prognosis

Question 9

What do systematic reviews of qualitative studies allow us to do?

Answer 9

To have a more comprehensive understanding of patients’ experience and views

Question 10

How are systematic reviews located?

Answer 10

Can be difficult due to large volume of research.

Cochrane Database of Systematic Reviews is the premier source for systematic reviews of intervention effectiveness

DARE: Database of abstracts of reviews of effects

Specialist evidence databases

Discipline-specific databases

Large biomedical databases (using a systematic review filter)

Question 11

How is a systematic review conducted?

Answer 11

Requires a significant and dedicated time commitment, varying depending on the number of citations/abstracts involved.

Involves many steps:

1. Formulate a question
2. Define eligibility criteria for the systematic review
3. Conduct the search for studies
4. Screen studies for inclusion
5. Review full-texts of eligible studies
6. Assess the risk of bias
7. Abstract the data
8. Meta-Analysis (if being done)

Question 12

What is included in a meta-analysis?

Answer 12

Generate estimates and Confidence Intervals

Rate confidence in estimates

Question 13

How is the research question defined?

Answer 13

Should be clearly focused and defined using PICO

Should make sense clinically and be worth answering

Question 14

When should decision be made about the methods for searching, screening, appraisal and synthesis?

Answer 14

Before commencing review.

Question 15

How are the eligibility criteria determined for a systematic review?

Answer 15

Using explicit, predefined criteria for including and excluding studies.

Eligibility criteria for systematic reviews of the effects of interventions have focused on defining types of participants, types of interventions and comparisons and types of studies.

Question 16

How are potentially eligible studies found?

Answer 16

Search methodology should be clearly explained and reproducible.

Multiple databases must be searched and citation tracking must be used.

Question 17

How is publication bias avoided in systematic reviews?

Answer 17

Locate studies that have not already been published and ongoing or planned studies in the area. (contact authors and doctoral theses) this is to avoid publication bias.

Question 18

After finding studies what is done to refine the studies to what is important?

Answer 18

The eligibility criteria set before are applied to selected studies to be included in the review.

At least 2 authors select papers.

Titles and abstracts are screened for relevance and eligibility

A final evaluation of the full-text articles is conducted and a decision is made on whether to include or exclude the studies.

Question 19

How is bias avoided in selecting papers?

Answer 19

During the refining process at least 2 authors must do the selection process to eliminate selection bias.

Question 20

How is the risk of bias assessed in a systematic review?

Answer 20

2 or more people independently extract data and judge the likely risk of bias.

A risk-of-bias tool is used.

Question 21

What information is extracted from included studies in a systematic review?

Answer 21

Participants

Intervention

Outcome measures used

Results of study

Study methodology and risk of bias

This is all standardised and used to summarize the data.

Question 22

What must be evaluated for a meta-analysis?

Answer 22

The intervention effect, including confidence intervals is calculated for each of the studies.

Overall intervention effect is calculated.

Question 23

Why may a meta-analysis not be possible?

Answer 23

Because the individual studies may not be similar enough in one or more respects.

Forest plots may still be presented but without an overall estimate and/or results are described narratively.

Question 24

What is a meta-analysis?

Answer 24

A statistical technique that enables data from multiple studies to be combined as a new dataset.

It includes larger sample size and more events than any one study.

Principle output is a forest plot.

Question 25

What does a forest plot tell us? What are its components?

Answer 25

The compiled data of several studies are put onto one big graph.

Each horizontal line with a square in the middle represents the results of a different study.

Square is called the point estimate (result of a particular study)

Size of the point estimates echoes the length of the confidence interval.

Diamond is called a summary estimate. (summary of the weighted results from the combined studies)

Vertical line in the center is the line of no effect (if result touches it or crosses it then the result is not statistically significant)

Question 26

What are the steps of critically appraising quantitative systematic reviews?

Answer 26

1. Methods: The validity of the review methodology
2. Results: The magnitude and precision of the intervention effect
3. Applicability: Of the review to your patient or population

A checklist can be used to guide this process. (eg CASP, AMSTAR)

Question 27

What are the reporting statements (checklists) used for quantitative and qualitative reviews?

Answer 27

Quantitative: PRISMA

Qualitative (ENTREQ)

Question 28

What do reporting statements contain?

Answer 28

PRISMA contains 27 steps for authors to ensure all important information is present.

ENTREQ uses a 21 step checklist