L22: Systematic review & meta-analysis

Question 1

What is a systematic review?

Answer 1

A review of a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant research and to collect and analyze data from the studies included in the review

Question 2

What is meta analysis?

Answer 2

The use of statistical techniques to integrate results of several independent studies into a single quantitative estimate or summary effect size.

Question 3

What is the difference between narrative review and systematic review?

Answer 3

Narrative review:

• often broad in scope
• sources and search usually not specified, potentially biased
• selection of studies not usually specified, potentially biased
• appraisal is variable
• often a qualitative summary

Systematic review:

• often a focused clinical question
• comprehensive sources and explicit search strategy
• criterion-based selection, uniformly applied
• rigorous critical appraisal
• may or may not include meta analysis

Question 4

What is the purpose of meta analysis?

Answer 4

• increase statistical power
• improve precision
• settle controversies arising from apparently conflicting studies or to generate new hypotheses
• answer questions not posed by individual studies

Question 5

How is systematic review relevant to pharmacy practice?

Answer 5

• development of evidence based practice guidelines

- economic evaluation in healthcare

Question 6

What are the six steps in conducting a systematic review?

Answer 6

1. Formulate review question (PICO approach) and develop systematic review protocol (study selection criteria - inclusion/exclusion criteria)
2. Search the literature (comprehensive literature search in multiple electronic databases, hand searching of reference lists of publications, formulate search terms, document and keep record of search strategy, limit to english language) ; study selection based on pre-defined selection criteria, record numbers of studies included and excluded at each step and reasons for exclusion (ideally at least two independent reviewers)
3. Assess study quality - at least two independent reviewers should be involved; Jadad scale used to assess quality of RCTs (however it is a composite score); Cochrane risk of bias tool to assess quality of RCTs; Newcastle-Ottawa Scale used to assess quality of observational studies specifically case-control and cohort studies
4. Abstract data - develop standardized data abstraction form to assess relevant study characteristics including quality aspects and all relevant results (ideally at least two reviewers should abstract information independently); discrepancies resolved by discussing or referral to third party
5. Analyze data (may include meta analysis) and interpret results
6. Report findings - summarize key findings

Question 7

How do we do qualitative data analysis?

Answer 7

tabulation/graphical display of characteristics and results of individual studies

Question 8

How do we do quantitative data analysis?

Answer 8

• use statistical software
• present as forest plot
• examine heterogeneity of studies
• assess for publication bias

Question 9

What does size of square represent in a forest plot?

Answer 9

The weight of the study in the meta-analysis (larger square -> larger sample size -> higher weight)

Question 10

What does the diamond represent in a forest plot?

Answer 10

• mid point of diamond -> summary effect measure (pooled result from all the studies)
• width of the diamond -> 95% CI of the summary measure

Question 11

How do we examine heterogeneity among studies?

Answer 11

Clinical heterogeneity:

• different treatment doses/duration/regimen
• differences in sample populations
• differences in timing and method of measuring outcome

Methodological heterogeneity:

• Type of study design
• study quality

Statistical heterogeneity:

• visual inspection of forest plot - do the CI of studies overlap with each other and the summary effect measure?
• Cochran’s Q test (chi square test) -> p<0.10 -> significant statistical heterogeneity
• I2 statistic (larger value -> higher heterogeneity)
• when there is concern about statistical heterogeneity, analyze by random-effects model.
• when there is no statistical heterogeneity, analyze by fixed-effects model

Question 12

How do we assess for publication bias?

Answer 12

• published studies tend to give positive results -> results from meta analysis may overstate a treatment effect
• visual inspection of funnel plot to assess (if asymmetrical -> publication bias)
• Statistical test: egger’s test (p<0.05 -> asymmetry -> presence of bias)

Question 13

What is a funnel plot?

Answer 13

size of study/precision (y axis) vs effect measure (x axis)

Question 14

What statement is used for reporting systematic reviews and meta analysis?

Answer 14

PRISMA statement