Systematic reviews Flashcards
Why do we need to review research?
So many articles, we need a good summary
What is a review?
Summary of what is known, often prepared by an expert and can be on any topic
What are the advantages of a traditional review? 3
Summarise everything Give flavor of a subject Might point us towards original research
What are the problems with a traditional review?4+
Messy Not clear Only sees part of the picture Unscientific- not standardised, no clear purpose, presents impressions of one individual
What do systemic reviews do?
Allow us to objectively review all available evidence Attempt to collate all evidence to answer a specific question using systemic ways More reliable
What do systemic reviews mostly look at?
effects of a treatment on a disease
Where does the best evidence for systemic reviews come from?
Randomised controlled trials
What type of data do most systemic reviews use?
Summary/aggregate data
What does a protocol for a systemic review need to have?6
Clear question with objectives set out search strategy to identify all trials consistent data collection across all trials Assessment of trial quality Synthesis of results Structured presentation of results
5 things systemic reviews should specify
PICOS Population Intervention Comparison Outcomes Study design
How to ensure the studies are similar
Set up good eligibility criteria Include trials with sim questions Some difference eg dosage expected
Where do you find studies: how
Rely on bibliographic data bases, build a structured search strategy and use index terms and free text terms
Biases involved with finding studies?
Studies with more dramatic results are more likely to be published, quicker and in better journals Publication bias, time lag bias, language bias –> reporting biases
How to avoid reporting biases?
All relevant trials published and unpublished checked
other than bibliographic data bases, where to look? 5
Grey literature Conference abstracts and proceedings Trial registers References for relevant articles Ask experts in fields
Which studies do you exclude?
Ineligible populations, interventions, comparisons and study types
How to collate data?
Using a predefined form
3 other challenges with obtaining enough data: what to do
Trial authors might not report results for some outcomes Not give detailed report of methods Not state exclusions –> ask for missing data
Why do you need to assess study quality?
Trials with poor design, conduct or analysis may over or underestimate effect of treatments
How to assess quality of studies?
Need to extract info to judge (methods of randomisation, analysis methods, outcomes)
A good systematic review will: 3
Provide a comprehensive objective and unbiased summary of all evidence Overcome problems with traditional reviews Results can be presented narratively as it will have a meta analysis
What is a meta analysis?
Quantitatively combining results of related trials to get an overall average effect of treatment
Why use meta analysis?3
To detect smaller effects reliably (they might still be very imp) Provides more patients than any one trial so greater power to detect difference between treatment and control and greater confidence that estimate of effects represent the truth
why do you need a systematic review with the meta analysis?
To prevent biased results
2 stages of meta analysis
Extract or calculate estimate of effects of each trial combine these to get overall estimate of effect
Formula for meta analysis
Meta analysis effect = Sum of (trial effects x weight)/ sum of weights
How is meta analysis results presented?
Forest plot
What are time to event outcomes?
Whether things do or dont happen over a period of time
How are time to event outcome events usually measured?
Hazard Ratio
What does hazard ratio
Benefit
How do you get relative effect of treatment in a time to event outcome?
1-HR
What is a continuous outcome?
Whether a disease or participant measure changes
How are continuous outcomes measured?
Mean difference or standard mean difference
Mean difference
benefit
Why would you get heterogeneity between trials?
Dif populations, treatments, trial designs…
When do you call it a statistical heterogeneity?
Difference is > expected by chance
How to test for heterogeneity?
Heterogeneity Chi2 I2 Stat
With heterogeneity chi2, what results could mean heterogeneity?
P
What unit does I2 stat work with?
Percentages
I2 stat results range show:
Low - low heterogeneity high - high heterogeneity
Difference between fixed effect model and random effects model:
Fixed assumes same effect in each trial and weights by trial size Random assumes normal distribution of effect around mean and weights by trial size and heterogeneity
When do people use random effects?
They use it to allow for any heterogeneity,
What does random effects do if moderate heterogeneity?
Small trials have more weight, the random effects confidence interval is wider, fixed and random effects may differ - explore how and why results vary
If no heterogeneity, what does random effects do?
if none, big trials have more weight and both fixed and random give same results
If there is substantial heterogeneity what do you get for chi2 and I2?
Very small chi 2 p value and I2 close to 100%
Fixed vs random effects if substantial heterogeneity?
Results vary loads
Problem if substantial heterogeneity:
Is meta analysis justified?
What questions should you ask when exploring heterogeneity of effects?
Do they vary by treatment or other trial characteristics? Do they vary by participant characteristics?
How to find out what the studies vary by?
Trial subgroup analysis or sensitivity analusis
How to figure out if the trials vary by participant characteristics:
Need data on each patient - IPD
How do we do a sensitivity analysis?example
Do meta analysis of all trials, then do meta analysis after excluding the two largest trials - how similar are the results?
How to do a trial subgroup analysis?
Group them by ie chemo regimen DO mini meta analysis for each subgroup Compare results
What types of data can meta analysis be based on?
Aggregate and individual participant data (IPD) - raw data
Aggregate data explain:
Often limited to published trials data common and quick
IPD explain:
Get raw data, validate and reanalyse Gold standard take longer and more resource intensive
How do you plan an analysis:
Analysis shouldn’t be driven by results: need a protocol including outcome measures, meta analysis models and how an effect varies by trial
When is meta analysis not appropriate?3
Effects are v heterogeneous Data insufficient or very variable Trials are of poor quality
What guidelines do you follow for systematic reviews?
PRISMA guidelines
What do PRISMA guidelines aim to do?
Improve reporting of systematic reviews and meta analysis help users of published systematic reviews critically appraise them
