Appraising Systematic Reviews

Question 1

What is a systematic review?

Answer 1

Method for of locating, appraising and synthesising research from primary studies

Question 2

How do systematic reviews differ from literature reviews?

Answer 2

prepared using predefined methods

- lit reviews don’t have strict requirements

Question 3

What does conducting a systematic review involve?

Answer 3

• research question
• determine eligibility criteria (pre-defined)
• comprehensive search to identify all relevant studies (beyond electronic searches)
• select studies based on predefined eligibility criteria
• appraise risk of bias in each study
• extract data
• synthesise results (meta-analysis)
• interpret and report results
• update review at an apropriate time

Question 4

What type of reviews overlap with meta-analysis?

Answer 4

systematic reviews
lit reviews (no strict method)

Question 5

Advantages of systematic review

Answer 5

Adv:

• established generalisability of overall results
• makes explicit differences in results
• identifies gaps in knowledge
• guides decision making
• improves dissemination of evidence
• hasten assimilation of research into practise

Question 6

Disadvantages of systematic review

Answer 6

can be done badly

Question 7

What is the best study design to assess effectiveness of intervention?

Answer 7

Systematic review of RCT

Question 8

What is the best source for systematic reviews

Answer 8

Cochrane Database

Question 9

What is so good about Cochrane systematic reviews?

Answer 9

• rigorous standards

- editorial teams support reviews

Question 10

Can systematic reviews be updated?

Answer 10

YES- as new studies are released

Question 11

What are the checklists we use to appraise systematic reviews?

Answer 11

AMSTAR (Assessment of multiple systematic reviews)

• CASP (checklist for appraising reviews)

Question 12

Criteria for systematic reviews

Answer 12

AMSTAR- check word doc

Question 13

9. Were the methods used to combine the findings of the study appropriate? what are you looking for?

Answer 13

I^2

Question 14

10. Was the likelihood of publication bias assessed? What are we looking for?

Answer 14

funnel plot or Egger regression test

Question 15

11. Was conflict of interest included?

Answer 15

acknowledged sources of support in systematic review and included studies
- can be in declarations of interest

Question 16

Most common way results are presented for meta-analysis

Answer 16

forest plot

Question 17

Interpreting funnel plot

box, line, size of box, diamond, points of edge of diamond

Answer 17

• Box = mean
• Line = confidence interval
• Size of box = weights
• Diamond = represents overall effect when combining info from the studies
• Points of edge of diamond = 95% confidence interval
• Spread of diamond = range

Question 18

1. Was an ‘a priori’ design provided?

Answer 18

inclusion criteria & research question established before conduct of review

Question 19

2. Was there duplicate study selection and data excration?

Answer 19

2 or more independent data extractors

Question 20

3. Was a comprehensive lit review performed?

Answer 20

2 or more electronic sources searched

Question 21

4. Was the status of publication (i.e. grey literature) used as an inclusion criteria?

Answer 21

want no exclusion based on publication status, language etc.
(no is good)

Question 22

5. Was there a list of studies provided?

Answer 22

• list of included and excluded studies provided

- Cochrane always include

Question 23

6. Were the characteristics of included studies provided?

Answer 23

data on participants, interventions, outcomes (often in table) included

Question 24

7. Was the scientific quality of included studies assessed and documented?

Answer 24

• critically appraise each study

- satisfied if ‘a priori’ methods of assessment provided

Question 25

8. Was the scientific quality of the included studies used appropriately in formulating conclusions?

Answer 25

putting less weight on studies prone to bias and more weight on studies performed well
e.g. “the results should be interpreted with caution due to poor quality of included studies.”