Systematic Review & Meta Analysis Flashcards
Hierarchy of Evidence (9)
- ideas, editorials, opinions
- Systematic reviews & meta analysis
- Randomised controlled double-blind trials
- Cohort studies
- Case-control studies
- Case series
- Case report
- Ideas, editorials, opinions
- Animal research
- In-vitro (test tube) research
Systematic review definition (4)
- a clearly formulated question
- uses systematic & explicit methods to identify, select & critically appraise relevant research
- to collect & analyse data from the studies that are included in the review
- may or may not use statistical methods to analyse & summarise the results (meta-analysis)
Meta analysis
- use statistical techniques to integrate the results of several independent studies into a single quantitative estimate or summary effect size
Difference between Narrative Review & Systematic Review
- Question
- Sources & search
- Selection of independent studies
- Appraisal
- Synthesis of data
Narrative Review
- Question
- broad in scope - Sources & search
- not specified
- potentially bias - Selection of independent studies
- not specified
- potentially bias - Appraisal
- variable - Synthesis of data
- often a qualitative summary
Systematic Review
- Question
- focused clinical question - Sources & search
- comprehensive sources & explicit search strategy - Selection
- uniform criterion-based selection - Appraisal
- rigorous critical appraisal - Synthesis of data
- may or may not include meta-analysis
Purpose of Meta analysis (4)
- Increase statistical power
- Improve precision
- increase sample size leads to narrower 95% CI - Settle controversies arising from apparently conflicting studies or to generate new hypothesis
- Answer questions not posed by the individual studies
Relevance of systematic review to pharmacy practice (2)
- Development of evidence-based practice guidelines
- Economic evaluations in healthcare
- generate clinical policies to optimise outcomes using available resources
eg cost-effectiveness
**
Steps in conducting systematic review
- with or without meta analysis
(6)
- *
1. Formulate the review question & develop the systematic review protocol
2. Search the literature - search strategy
- study selection
3. Assess quality of study
4. Abstract data
5. Analyse data (may include meta analysis) & interpret results - heterogeneity among studies (clinical, methodological & statistical)
- publication bias
6. Report findings
How to formulate the review question?
- use PICO approach
Population
Intervention
Comparison
Outcome
Develop the systematic review protocol
- criteria to selection of studies
- define the study selection criteria (inclusion & exclusion criteria)
Inclusion / exclusion criteria
- type of study design
- type of population
- type of intervention
- type of comparison
- type of outcome
(( PICO ))
Search strategy
- conduct comprehensive literature search
eg multiple electronic databases or hand searching of reference lists of publications - formulate search terms for electronic search
Examples of multiple electronic databases (3)
- PubMed
- Embase
- Cochrane Central Register of Controlled Trials
Study selection process (4)
- based on pre-defined study selection criteria
- record the number of studies included & excluded at each step & reason for exclusion
- at least 2 independent reviewers to select studies
- any discrepancies between reviewers to be resolved through discussion or third investigator
Assess quality of study (3)
- at least 2 independent reviewers to be involved
- any discrepancies between reviewers to be resolved through discussion or third investigator
- 3 types
Types of scale to assess quality of study (2+1)
For RCT
- Jaded scale
- Cochrane risk-of-bias tool ver 2
For observational studies
1. Newcastle-Ottawa Scale (NOS)
Jaded scale (2)
- RCT
- score range from 0-5
- low (0-2), high (3-5)
Disadvantage of Jaded scale
- score cannot tell which part of the study is lacking unless table is reported
Cochrane RoB tool ver 2 (2)
- RCT
- domain based (low risk of bias, some concern & high risk of bias)
Newcastle-Ottawa Scale (NOS)
- observational studies (specifically case-control & cohort studies)
NOS for case-control studies
- selection
- comparability
- ascertainment
- selection of cases & controls
- comparability of cases & control
- ascertainment of exposure
NOS for cohort studies
- selection
- comparability
- ascertainment
- selection of exposed & unexposed
- comparability of exposed & unexposed
- ascertainment of outcome
Abstract data
- develop a standardised data abstraction form
- at least 2 reviewers to abstract information from studies independently
- any discrepancies between reviewers to be resolved through discussion or third investigator
Methods to reduce discrepancies in
- study selection
- assessment of study quality
- abstract data
(2)
- discussion between reviewers until consensus is reached
- invite third investigator for review & resolution
Types of analysis of data (2)
- Qualitative data synthesis
2. Quantitative data synthesis
Qualitative data synthesis
- tabulation / graphical display of characteristics & results of individual studies
Quantitative data synthesis
& uses
(3)
- use statistical software eg Sata, RevMan - examine heterogeneity among studies eg Forest plot - assess for publication biass eg Funnel plot
Forest plot
- square (midpoint)
- line
- size of square
- diamond (midpoint)
- width of diamond
- each tree (line) represents a study
- square is the point estimate
- size of square is weight of the study
- horizontal line is 95% CI
- mid-point of diamond is summary effect measure
- width of diamond is 95% CI of summary measure
Weight of the study
= (1 / variance)
- large sample size, smaller variance, larger weight
- small sample size, larger variance, smaller weight
Types of heterogeneity among studies (3)
& its stratification
- Clinical heterogeneity
- clinical relevance
eg doses, duration of treatment, population type, method of measuring outcome - Methodological heterogeneity
- study design & quality
eg case-control, cohort, RCT - Statistical heterogeneity
- statistical methods
Statistical heterogeneity among studies
How to assess for statistical heterogeneity among studies (2)
- Cochran’s Q test
2. I^2 statistic
Cochran’s Q test
- chi square test
- statistically significant heterogeneity if p<0.1
I^2 statistic
- proportion of total variance due to between study variability
- in terms of %
- increasing value means increasing heterogeneity
= (between study variability/total variance) x 100%
If there is statistical significant heterogeneity among studies, what model to use? (2)
- use random-effects model
- to account for variability between studies
If there is no statistical significant heterogeneity among studies, what model to use? (1)
- use fixed-effects model
Assessment for publication bias (2)
- use funnel plot
- Egger’s test for funnel plot asymmetry, asymmetry if p<0.05 suggesting presence of bias
What type of analysis is prone to publication bias?
Why? (2)
Meta analysis because :
- studies with negative results are less likely to be published & mainly positive results are published & hence used in systemic review
- hence, results from meta-analysis may overstate a treatment effect (or reduce impact of exposure)
Funnel plot
- x & y axis
- description of plot (4)
(6)
- y axis : size of study or precision/standard error
- x axis : effect measure
eg risk ratio, odds ratio - smaller study have wider scatter of effect estimate (bottom of pyramid)
- larger study have narrower scatter of effect estimate
(top of pyramid) - top of pyramid is the line of symmetry
- slope of both sides of the pyramid is 95% CI of overall effect estimate
** Report findings (4)
**
- summarise key findings
- judge validity of findings
eg poor quality studies can recommend to disregard
- judge for generalisability
- implications for further studies or clinical practice
Reporting format for systematic reviews & meta analysis (2)
- PRISMA statement
2. MOOSE guidelines
PRISMA statement
Preferred Reporting Items for Systematic reviews & Meta Analysis
MOOSE guidelines
Meta analysis Of Observational Studies in Epidemiology
Reporting guidelines
- observational studies
- systematic review & meta analysis
Observational studies
- STROBE (STrengthening the Reporting of OBservational studies & Epidemeoogy)
Systematic review & meta analysis
- PRISMA (Preferred Reporting Items for Systematic reviews & Meta Analyses)
- MOOSE (Meta analysis Of Observational Studies & Epidemiology)
