Session 6: Reviews of Evidence Flashcards
Features of narrative reviews.
Implicit assumptions
Opaque methodology
Not reproducible leading to bias and subjectivity
Give features of systematic reviews.
A clearly focused question
An explicity statement about type of study, type of participants, types of interventions, types of outcome measures.
A systematic literature search
A selection of materials
Appraisal
A synthesis possible including a meta-analysis.
Key features of systematic reviews.
Explicit assumptions
Transparent methodology
Reproducible leading to unbias and objectivity
What is a meta-analysis?
A quantitative synthesis of the results of two or more primary studies that addressed the same hypothesis in the same way.
Difference between systematic review and meta-analysis.
A systematic review will not necessarily include a meta-analysis but more or less all MA have systematic reviews.
Give an example of when systematic review won’t include MA.
When clinical heterogeneity is too great
Purpose of meta-analysis.
Facilitate synthesis of a large number of study results.
Systematically collate study results
Reduce problems of interpretation due to variations in sampling
Quality criteria of MA.
Compilation of complete set of studies
Identification of common variable or category definition.
Standardised data extraction
Analysis allowing for sources of variation
Calculate the odds ratio of surviving.
Aspirin: 566/49 = 11.55:1
Placebo: 557/67 = 8.31:1
Odds ratio = 11.55/8.31 = 1.39
Interpret the result of the odds ratio being 1.39.
1.39 more likely to survive if you are on aspirin then on placebo.
The odds ratio is 1.39 and the error factor is 1.48.
Calculate the confidence interval.
- 39 x 1.48 = 2.05
- 39/1.48 = 0.94
The confidence interval is 0.94 - 2.05
The confidence interval is 0.94 - 2.05
Is the result statistically significant?
No.
The null hypothesis (1) is within the confidence interval which means that the result could be due to chance.
Meta-analysis calculated a pooled estimate of odds ratio.
Explain what this means.
Odds ratio and their 95% CIs are calculated for each study.
They are then combined to give a pooled estimate odds ratio using a statistical computer program.
In the pooled estimate the individual studies are weighted.
Weighted according to what?
Their size
The uncertainty of their odds ratio
What does a large size of a study suggest in weight?
Greater weight
What does a narrow CI suggest according to weight?
A narrow CI = greater weight
How are meta-analysis visualised?
By a forest plot.
What does the size of the square indicate?
Weight of the study
What does the line going through each square indicate?
The CI of that study
What does the vertical line indicate?
The null hypothesis
What does the width of the diamond indicate?
The confidence interval of the pooled estimate
What does the vertical dotted line indicate?
The pooled odds ratio
Problems with meta-analysis
Heterogeneity between studies
Variable quality of the studies
Publication bias in selection of the studies
What are the two models in order to take heterogeneity into account?
Fixed effect model
Random effect model
Explain fixed effect model
Assumes that the studies are estimating exactly the same true effect size.
There is only one true effect in fixed effect model
Explain random effect model.
Assumes that the studies are estimating similar, but not the same true effect size.
There is mean true effect
Pooled estimate OR in fixed effect model was 1.11.
Pooled estimate OR in random effects model was 1.14.
The between study variance (t2) was 0.01.
What does this mean?
That the heterogeneity between the studies was low.
How does the odds ratio differ in fixed vs random effect?
It is often very similar.
How does the 95% CI differ in fixed vs random effect?
Often wide in random effect vs fixed effect.
How does the weighting of the studies differ in random effects vs fixed effect model?
More equal between studies in the random effects model which means that there is greater weighting towards small studies.
A test for heterogeneity has a p value of 0.082.
The p-value is regarded significant if the p-value is less than 0.1.
What does this mean?
This means that we can reject the null-hypothesis which is that there is heterogeneity.
In other words this means that there is homogeneity.
Limitations of random effects model and heterogeneity.
Random effects model can only account for variation but not explain it.
What is used to explain heterogeneity?
Sub-group analysis
Give examples of variable quality.
Poor study design
Poor design protocol
Poor protocol implementation
Rank studies in how prone they are to bias. (More prone higher up)
Case-control
Cohort
Non-randomised controlled trials
Randomised controlled trials
Approaches to analyse variable quality in studies.
Define a basic quality standard and only include studies that satisfy the criteria.
Score each study for its quality and then incorporate the quality score into weighting, or use sub-group analyses to explore differences (high quality studies vs. low quality studies)
Give examples of what to assess in RCTs.
Allocation methods (was it randomised?)
Blinding and outcome assessment
Intention-To-Treat
Appropriate statistical analysis
What is publication bias?
Studies with statistically significant or favourable restults are more likely to be published than those studies with non-statistically significant or unfavourable results.
This applies particularly to smaller studies.
What does publication bias mean in terms of systematic review or meta-analysis?
That they can be flawed by such bias.
Publication bias leads to a biased selection of studies towards demonstration of effect.
What is used in order to identify whether there is publication bias or not?
Plot results of identified studies against a measure of their size via a funnel plot.
