systematic review and meta analysis Flashcards
What is involved in a systematic review
review all info available concerning a clearly formulated question, use systematic and explicit methods to identify data, select relevant data, critically appraise research ie include/exclude, collect data from the studies, analyse data
range of qualities of reviews and evidence in literature - summarise and process which info is reliable. critical method for assessing this (critical appraisal)
Why conduct a systematic review
data quantity - number of citations in pubmed increases, because of more publications and more open access journals - you can produce as many articles as you like. Increase because of globalisation, internet and open access. new journal every day. Not feasible as a clinician to read them all
data quality: individual study/case report - unable to conclusively answer a business question, poor study design/small numbers - low power = false -ve results, difficult to generalises. Multiple studies: which source to trust when many are different/diverging, how to compare different studies using different protocols
How do you find systematic reviews
Cochrane - database of systematic reviews, about 8000, range of protocols. reviews updated on new evidence, active resource up to date
Stages in a systematic review
1 - planning review - critical, save time down the line
2a - identification of research
2b - selection of studies - assess volume and select ones interested in
2c - quality assessment
3a - data analysis
3b - data visualisation
3c - reporting and dissemination
planning the review
specify question to be addressed - eg should I recommend the flu jab
what was the focus of the review
find info I the summary
P opulation - all ages/one subgroup eg pregnant women and effect on new born
I ntervention/
C omparison - all methods of administration, there are different ways to give a vaccine and 3 different types
O utcomes - clinical - symptomatic infuenza and influenza like illness: number of cases side effects/days of work lost/specific outcome for pregnancy. Harms - serious systemic adverse effects eg nausea and fever. maternal outcomes -abortion, still birth, preterm birth, maternal death. neonatal - congenital malformations, death. secondary local adverse effects eg redness at site of injection
S tudy designs - RCT and quasi-RCT - compare placebo and vaccine, or different types of vaccine. comparative non-randomised study - efficacy in pregnancy or specific syndromes like Guillain-Barre
identification of research
need to decide in advance when stop getting information
if studies are missing it creates bias
remove duplicates
have inclusion and exclusion criteria
have 2 people doing it - minimise chance of subjectivity and error
quality
clearly defined search criteria - MeSH
search published medical literature - electronic database like Cochrane, Medline and EMBASE
search other sources - reference list citations, conference proceedings, contact researchers to get unpublished studies
Selection of studies
inclusion and exclusion criteria
extract data using data extraction form - methodological quality of studies: study design, description of setting, characteristics, description of vaccines, description of outcomes, publication status, date of study, location of study.
2 people
Study quality assessment
assessed by recognised/user defined criteria
assess biases in study design including selection bias, measurement bias, attrition bias/loss to follow up
preferably assessed before the study results are known
ideally with more than 1 assessor
traffic light rating for bias
Data analysis
odd ratio, relative risk and confidence interval
forest plot - bar show risk left favours vaccine, right risk of vaccine, diamond is the summary of all the evidence
confidence interval - smaller if there is a larger study
What is a meta- analysis
use of statistical techniques in a systematic review to integrate results of included studies
the studies are the primary units of analysis
sub part of review
larger study - has more weight it should have
software helps
generate forest plot
PRIMSA - example of guidelines
Reporting and dissemination
study details tabulated in a meaningful way
should include details of PICOS
includes summary of findings
limitations of systematic reviews and meta-analysis
depend on the amount of evidence that you can find
bias
inconsistency of results - heterogeneity
low study quality
incompleteness of review - ie not all studies included
low number of studies
lack of generalisability
publication bias
data might be in a different language - might not get all sources
give higher/lower odds ratio than expected
studies may not be representative of all valid studies
funnel plot - each study is a point, position determined by result, odds ratio and precision (sample size) no bias = symmetrical about mean, bias: asymmetric - miss lower R/L hand corner
Heterogeneity
inconsistency with study results
sub studies have smaller groups and so weaker evidence
studies differ with respect to: PICOS, clinical differences, methodology differences and unknown study characteristics
I squared give sense of heterogeneity 0= no, 25=low 50 = moderate, 75=high heterogeneity
can ask authors about bias
to explore heterogeneity look at subgroups to see if results differ, meta-regression, sensitivity analysis
Advantages of systematic reviews and meta analysis
pooled overall risk estimate
more reliable and precise estimate of effect
explore differences between studies
identify pub bias
how do you critically appraise a systematic review
was a clear qn predefined - PICOS
comprehensive search for literature carried out - databases, grey literature, inclusions/exclusions, duplicate and independent assessment of literature
methodological quality assessed appropriately - quality assessed as inclusion criteria, measures appropriate, heterogeneity due to quality
heterogeneity explored due to PICOS
how credible is the evidence - strengths and weaknesses, high quality studies, impact on clinical practice, applicability to other populations
check guidelines for reporting
how do you critically appraise a meta-analysis
same as systematic review but also
heterogeneity - PICOS
pub bias - missing studies
was it appropriate to pool the studies - sufficiently homogenous for studies to be pooled
appropriate model used to pool estimates - fixed v random effects model
did different sub-groups of studies give similar results - how generalisable, new hypothesis that should be explored
