S6) Reviews of Evidence Flashcards
Literature reviews of studies are used in evidence-based healthcare.
Describe the difference between these studies
- Narrative reviews: implicit assumptions, opaque methodology, not reproducible → biased, subjective
- Systematic reviews: explicit assumptions, transparent methodology, reproducible → unbiased, objective
Systematic reviews involve a clearly focused question with explicit statements about which factors?
- Types of study
- Types of participants
- Types of interventions
- Types of outcome measures
What is a systematic review?
A systematic review is an overview of primary studies that used explicit and reproducible methods, very credible, explicit, transparent and reproducible
Identify the four steps involved in a systematic review
⇒ Systematic literature search
⇒ Selection of the materials
⇒ Appraisal
⇒ Synthesis
What is a meta-analysis?
A meta-analysis is a quantitative synthesis of the results of two or more primary studies that addressed the same hypothesis in the same way
Illustrate the relationship between a systematic review and a meta-analysis
A systematic review will not necessarily include a meta-analysis if, for example, clinical heterogeneity is too great but a meta analysis can’t exit without being part of a systematic review
State the four purposes of a meta-analysis
- To facilitate the synthesis of a large number of study results
- To systematically collate study results
- To reduce problems of interpretation due to variations in sampling
- To quantify effect sizes and their uncertainty as a pooled estimate
To ensure quality criteria, a meta-analysis should have a formal protocol.
What should be specified?
- Compilation of complete set of studies
- Identification of common variable or category definition
- Standardised data extraction
- Analysis allowing for sources of variation
How does one interpret a forest plot?
- Individual odds ratios (squares) with their 95% CI (lines) are displayed for each study
- Size of square is in proportion to the weight given to the study
- The (diamond) is the pooled estimate with the centre indicating the pooled odds ratio (dotted line) and the width representing the pooled 95% CI
- The (solid line) is the null hypothesis OR
= here 6/7 had odds ratio>1 showing greater rate of survival amongst patients taking aspirin after MI
= the largest square was there only one with a statistically significant result but its 0R Was less Thant he others
What are the three major problems with a meta-analysis?
- Heterogeneity between studies
- Variable quality of the studies
- Publication bias in selection of studie
There are two approaches to calculating the pooled estimate odds ratio (OR) and its 95% CI.
Describe both the fixed effect model and the random effects model
these can remove the heterogeneity in the models
- Fixed effect model – assumes that the studies are estimating exactly the same true effect size
- Random effects model – assumes that the studies are estimating similar, but not the same, true effect size
Compare and contrast the fixed effect model with the random effects model in light of the following:
- Point estimate
- 95% Confidence Interval
- Weighting of the studies
- Hypothesis test for heterogeneity
- Point estimate e.g. odds ratio – is often similar in both models
- 95% CI – is often wider in the Random Effects Model
- Weighting of the studies – is more equal between the studies in the Random Effects Model i.e. greater weighting towards small studies
- Hypothesis test for heterogeneity – low statistical power to detect heterogeneity, often use 10% significance level
What are the causes for variable quality?
- Poor study design
- Poor design protocol
- Poor protocol implementation
What is the reason for publication bias?
Studies with statistically significant / ‘favourable’ results are more likely to be published than those studies with non-statistically significant / ‘unfavourable’ results
What are the consequences of publication bias?
- Any systematic review / meta-analysis can be flawed by such bias
- Publication bias leads to a biased selection of studies towards demonstration of effect
How might one interpret a funnel plot for publication bias?
- If no publication bias, then the plot will be a ‘balanced’/symmetrical funnel
- Smaller studies can be expected to vary further from the ‘central’ effect size
- Publication bias is likely to exist if there are few small studies with results indicating small or ‘negative’ measure of effect
key points
