S6 Systematic Reviews

Question 1

Evidence-based healthcare

Answer 1

Healthcare services should be based on best available evidence.
Literature reviews of studies;
• Narrative reviews: implicit assumptions, not reproducible ⇒biased
• Systematic reviews: explicit assumptions, transparent, methodology, reproducible ⇒unbiased

Question 2

Systematic reviews

Answer 2

• An overview of primary studies that used explicit and reproducible methods
• Gives explicit statements about the types of study, participants and outcome measures.
• Four steps: Systematic literature search, Selection of the materials, Appraisal and Synthesis
• SR are an extremely credible source of evidence as they are explicit, transparent and reproducible
• SR usually includes a MA, but not always e.g if clinical heterogeneity is too great.

Question 3

Meta-analysis

Answer 3

• A quantitative synthesis of results of two or more primary studies that addressed the same hypothesis in the same way
• Purpose: To facilitate the synthesis of a large number of study results. To reduce problems of interpretation. To quantify effect sizes and their uncertainty as a pooled estimate
• MA should have a formal protocol specifying: compilation of complete set of studies, standardised data extraction and analysis
• Problems: Heterogeneity between studies, Variable quality of studies, Publication bias in selection of studies

Question 4

Calculating odds ratio

Answer 4

95% CI indicates that the null hypothesis OR (= 1.00) is within its range, p>0.05 and so the results are not statistically significant and could be due to chance.
OR and their 95% CIs are calculated for all studies in meta-analysis, these are then combined to give a pooled estimate odds ratio using a statistical computer program. Studies are weighted according to their size; narrower CI → greater weight.

Question 5

Forest plot

Answer 5

• Squares are the OR (larger sq for larger weight), diamond is the pooled estimate, dotted line is pooled OR, solid line is null hypothesis OR

Question 6

Heterogeneity (diversity) between Studies

Answer 6

Two approaches to calculating the pooled estimate OR and its 95% CI:

Question 7

• Fixed effect model

Answer 7

assumes the studies are estimating the same true effect size

Question 8

• Random effects model

Answer 8

assumes the studies are estimating similar true effect size
o Point estimate e.g OR – similar in both models
o 95% CI – wider in RE
o Weighting of the studies – more equal between studies in the RE Model, i.e. greater weighting towards small studies
o Hypothesis test for heterogeneity – low statistical power often use 10% significance level

Question 9

Sub-Group Analysis

Answer 9

can be Stratification by study characteristics or by participant profile.

Question 10

Causes of variable quality:

Answer 10

poor study design, poor design protocol, poor protocol implementation.

Question 11

Approaches to variation

Answer 11

• Have a basic quality standard and only include studies satisfying this criteria
• Score each study for its quality and incorporate this into the weighting
RCT are the most prone to bias, case-control studies least.

Question 12

Publication Bias in Selection of Studies

Answer 12

• Reason for PB: Studies with statistically significant /‘favourable’ results are more likely to be published than those studies with ‘unfavourable’ results
• Consequences: systematic review or meta-analysis can be flawed by PB
• Methods of Identification: Check meta-analysis protocol for method of identification of studies, Plot against a measure of size e.g funnel plot, Use a statistical test for PB
• Funnel plot interpretation: If no publication bias, then the plot will be a symmetrical funnel, smaller studies vary further from central effect size.