11 Systematic reviews and Meta-analysis

Question 1

Q: Compare a systematic review and a meta-analysis.

Answer 1

A: A systematic review aims to answer a defined research question by collecting and summarising empirical evidence and critically appraise relevant research – usually published in the scientific literature - that fits pre-specified eligibility criteria. Looking at more than one study

Meta-analysis refers to statistical techniques used in a systematic review to integrate the results of studies matching the eligibility criteria

Question 2

Q: Why undertake a systematic review? Problem?

Answer 2

A: provide more generalised conclusions, researchers can conduct a systematic review of the primary studies on a particular research question to provide a comprehensive summary of our knowledge at the time of the review

multiple studies of the same research question often lead to inconsistent or even opposite results = problem

Question 3

Q: What does assessing quantity and quality of data require?

Answer 3

A: Efficient searching of data (i.e. literature)

Applying formal rules for critical appraisal of data

Question 4

Q: Systematic approach. Process? Meta-analysis advantage? Systematic review advantage?

Answer 4

A: – Transparent process because of the explicit methods in identifying and rejecting studies.
– A meta-analysis, if appropriate, will increase the power of the study and enhance the precision of estimates of treatment effects, accounting for sample size, and uncertainties.
– Systematic reviews may demonstrate the lack of adequate evidence and thus identify areas where further studies are needed

Question 5

Q: What do most scientific journals require from systematic reviews?

Answer 5

A: each stage to rigorously follow guidelines or “best practice” recommendations

Question 6

Q: Describe stage I of a systematic review. Framed around? (4)

Answer 6

A: – Planning the review : The authors need to clearly define the research question to be addressed
– This question is usually framed around the definition of study participants, intervention (exposure), outcomes and study designs of interest

Question 7

Q: Describe stage II of a systematic review. Requires? Selection? Study quality?

Answer 7

A: – Identification of research : This requires clearly defined search criteria and a thorough search of all published literature (including exhaustive searches of reference lists, conference proceedings and contact with researchers in the field).
– Selection of studies : Inclusion and exclusion criteria should be defined a priori; these are likely to be based on factors such as study design, year, sample size, completeness of information, study quality etc.
– Study quality assessment : Study quality can be assessed against recognised or user-defined criteria, usually to establish whether various biases are likely to exist in the in study (e.g. selection bias, measurement bias, attrition bias/loss to follow-up).

Question 8

Q: Describe stage III of a systematic review. (3)

Answer 8

A: – Reporting and dissemination – Study details need to be abstracted from each eligible study along with the effect estimate (or details that allow an effect estimate to be calculated)
– These details need to be tabulated in a meaningful way, including, where appropriate, details of populations, interventions/exposure, outcomes and study design, and a summary of the findings
– The last step consists in estimating an overall effect by combining the data

Question 9

Q: What is specific about meta-analysis when it comes to ‘subjects’? Benefit?

Answer 9

A: More subjects can be included than any single constituent study, producing a more reliable and precise estimate of effect

Question 10

Q: What are the advantages of a meta-analysis? (3)

Answer 10

A:  Produce a more reliable and precise estimate of effect
 Explore differences (heterogeneity) between published studies
 Identify whether a publication bias is occurring

Question 11

Q: What happens if the studies are too heterogeneous?

Answer 11

A: it may be inappropriate, even misleading to statistically pool the results from separate studies

Question 12

Q: What are the limiations of a meta-analysis? (4)

Answer 12

A:  Publication bias
 Labour intensive
 Inconsistency of results (studies differs with respect to, Populations Interventions/exposure, Outcomes, Study design, Clinical differences, Methodological differences, Unknown study characteristics)
 Low study quality

Question 13

Q: What is the most common way of visually summarising the results of a meta-analysis?

Answer 13

A: A Forest plot

This is a graphical representation of the results from each study together with the combined meta-analysis result

Question 14

Q: What is publication bias?

Answer 14

A: refers to the greater likelihood of research with statistically significant results to be published in the peer-reviewed literature in comparison to those with null or non-significant results

Question 15

Q: What can failure to include all relevant data in a meta-analysis lead to?

Answer 15

A: the effect of an intervention/exposure is over- or under-estimated

Question 16

Q: How can heterogeneity be explored?

Answer 16

A: Galbraith (radial) plots

Question 17

Q: Why may heterogeneity exist?

Answer 17

A: clinical differences, methodological differences or unknown study characteristics

studies may still differ with respect to the exact population, interventions/exposure, outcomes and designs used

Question 18

Q: What are the limitations in conducting systematic reviews? (3)

Answer 18

A:  If there are too few studies matching the eligibility criteria defined, a systematic review might not add much to the field
 If the methodological quality of studies is inadequate, then the findings of reviews of this material may also be compromised
 Publication bias can distort findings because studies with statistically significant results are more likely to get published

Question 19

Q: (Guidelines and protocols) What did Cochrane start as?

As a result?

Answer 19

A: organisation involving international researchers and clinicians -> organise medical research information in a systematic way, in order to facilitate the choices that health professionals, patients, policy makers and others face in health interventions according to the principles of evidence-based medicine

set many standards in the field

Question 20

Q: What is the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)? Focuses on?

Answer 20

A: an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses adopted by many scientific journals

PRISMA focuses on the reporting of reviews evaluating randomized trials, but can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions

Question 21

Q: What should you ask when appraising a systematic review? (5) Check?

Answer 21

A: Was a clear, pre-defined question addressed?

Was a comprehensive search for relevant research carried out?

Was methodological quality of each study assessed appropriately?

Was heterogeneity (consistency of results) explored?

How credible is the evidence?

(check guidelines for reporting (PRISMA))

Question 22

Q: What should you ask when appraising a meta-analysis? (5)

Answer 22

A: Was heterogeneity explored?

Was publication bias an issue?

Was it appropriate to pool studies?

Was the appropriate model used to pool effect estimates?

Did different sub groups of studies give similar results?

Question 23

Q: What do single studies rarely provide?

Answer 23

A: a conclusive, universal answer to a question

Question 24

Q: What can systematic reviews provide?

Answer 24

A: overview of evidence on a particular topic

Question 25

Q: What can meta-analyses provide? (3)

Answer 25

A:  A single, more precise, estimate of intervention/exposure effect (pooled risk estimate)
 A greater understanding of similarities/differences among studies
 An assessment of likely publication bias

Question 26

Q: What can systematic reviews/meta-analyses provide?

Answer 26

A: an evidence-base for clinical decisions