11 - Systematic reviews and meta- analysis Flashcards
Systematic reviews
answer a defined research question by collecting and summarising empirical evidence – usually published in the scientific literature – that fits prespecified eligibility criteria
‘a review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyse data from the studies that are included in the review’
Meta-analysis
statistical techniques used in a systematic review to integrate the results of studies matching the eligibility criteria.
Meta-analyses combine the published estimates of effect from each study to generate a pooled risk estimate
Advantages of systematic approach
Transparent process because of the explicit methods in identifying and rejecting studies.
A meta-analysis, if appropriate, will increase the power of the study and enhance the precision of estimates of treatment effects, accounting for sample size, and uncertainties.
Systematic reviews may demonstrate the lack of adequate evidence and thus identify areas where further studies are needed.
Systematic Review - Stage I
Planning the review – authors need to clearly define the research question to be addressed. This question is usually framed around the definition of study participants, intervention (exposure), outcomes and study designs of interest.
Systematic Review - Stage II
Identification of research – requires clearly defined search criteria and a thorough search of all published literature (including exhaustive searches of reference lists, conference proceedings and contact with researchers in the field).
Selection of studies – Inclusion and exclusion criteria should be defined a priority; these are likely to be based on factors such as study design, year, sample size, completeness of information, study quality etc.
Study quality assessment – Study quality can be assessed against recognised or user-defined criteria, usually to establish whether various biases are likely to exist in the study (e.g. selection bias, measurement bias, attrition bias/loss to followup).
Systematic Review - Stage III
Reporting and dissemination – Study details need to be abstracted from each eligible study along with the effect estimate (or details that allow an effect estimate to be calculated). These details need to be tabulated in a meaningful way, including, where appropriate, details of populations, interventions/exposure, outcomes and study design, and a summary of the findings. The last step consists in estimating an overall effect by combining the data, if a meta-analysis is deemed appropriate.
Advantages of Meta-Analysis
More subjects can be included than any single constituent study, producing a more reliable and precise estimate of effect
Generate a pooled overall risk estimate
Produce a more reliable and precise estimate of effect
Explore differences(heterogeneity) between published studies
Identify whether a publication bias is occurring
Limitations of Meta-Analysis
Publication bias
Labour intensive
Inconsistency of results (studies differs with respect to,
Populations Interventions/exposure, Outcomes, Study design, Clinical differences, Methodological differences, Unknown study characteristics)
Low study quality
Meta-analysis: statistics and visualization
Using studies included in the systematic review, effect estimates are abstracted (or calculated) from the selected studies; in a meta-analysis, these individual study effect estimates are then pooled to produce a weighted average effect across all studies.
A Forest plot (Box 1) is the most common way of visually summarizing the results of a meta-analysis. This is a graphical representation of the results from each study included in a systematic review, together with the combined meta-analysis result.
Each box is represented by a box and line - the size of the box corresponds to the weight given to that individual study (e.g. sample size); the horizontal lines correspond to the 95% confidence interval.
The overall estimate from the meta-analysis is usually shown as a diamond at the bottom of the plot. The centre of the diamond and dashed line corresponds to the summary effect estimate; the width of the diamond represents the confidence interval around this estimate.
How can publication bias in meta analysis be explored?
using Funnel plots, which show whether there is a link between study size (or precision) and the effect estimate
How can heterogeneity be explored?
Galbraith (radial) plots
Publication bias
Publication bias refers to the greater likelihood of research with statistically significant results to be published in the peer-reviewed literature in comparison to those with null or non-significant results. Failure to include all relevant data in a meta-analysis may mean the effect of an intervention/exposure is over- or under-estimated.
Limitations in conducting systematic reviews
▪ If there are too few studies matching the eligibility criteria defined, a systematic review might not add much to the field.
▪ If the methodological quality of studies is inadequate, then the findings of reviews of this material may also be compromised.
▪ Publication bias can distort findings because studies with statistically significant results are more likely to get published.
Cochrane
▪ started as an organization involving a large number of international researchers and clinicians to organize medical research information in a systematic way, in order to facilitate the choices that health professionals, patients, policy makers and others face in health interventions according to the principles of evidence-based medicine.
▪ Protocols developed and the Cochrane Database of Systematic Reviews - a database of systematic reviews and meta-analyses – set many standards in the field.
▪ Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses adopted by many scientific journals.
▪ PRISMA focuses on the reporting of reviews evaluating randomized trials, but can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions.
