Clinical epidemiology: Systematic reviews of RCTs Flashcards
What are some reasons why there are differences between RCTs?
Differences:
- in people compared
- in care, other than that being investigated
- in the way treatment effects are assessed
- in follow up
Available evidence:
- differences in reporting
- biased selection
Also obsolescence over time, changes in risk profile of patients, missing information
Hopefully not:
- bad science
- fraudulent science
What is a systematic review?
Review of all literature on one particular topic using scientific methods:
- clear question being addressed - PICO
- explicit and detailed statement of the methods used
- comprehensive ascertainment of the literature relevant to the review question
- steps to reduce bias in the review process with or without: a statistical technique used to produce a numerical summary of the size of effect with confidence intervals
What are the advantages of systematic reviews?
- can be the best source of evidence on the effects and effectiveness of healthcare interventions
- assimilate large amounts of research evidence
- provide reliable unbiased estimates of effect
- increase precision of estimates of effects
- provide information about the generalisability and consistency of effects
- Identify what information is missing
- Useful for making decisions
How does the purpose of a narrative review (non-systematic review) differ from a systematic review?
Narrative review:
- background information and current thinking
Systematic review:
- identify, evaluate and summarise current evidence
How do the methods of a narrative review (non-systematic review) differ from a systematic review?
Narrative review:
- Not specified
Systematic review:
- specified and reproducible
How does the review question of a narrative review (non-systematic review) differ from a systematic review?
Narrative review:
- often general discussion with no stated hypothesis
Systematic review:
- starts with clear question or hypothesis to be tested
How does the identification of studies of a narrative review (non-systematic review) differ from a systematic review?
Narrative review:
- well known articles, filling cabinet or selected e.g. medline only
Systematic review:
- planned search strategy for published and unpublished studies
How does the study selection for inclusion of a narrative review (non-systematic review) differ from a systematic review?
Narrative review:
- do not usually describe why studies are included/excluded
Systematic review:
- explicit criteria for included and excluded studies
How does the quality of assessment and data extraction of a narrative review (non-systematic review) differ from a systematic review?
Narrative review:
- subjective data extraction, do not usually consider study methods and bias
Systematic review:
- objective extraction of data, critical examination of study methods and bias -standard quality assessment method
How does the analysis of results of a narrative review (non-systematic review) differ from a systematic review?
Narrative review:
- various; often don’t consider differences between studies
Systematic review:
- Validated methods, assess heterogeneity, meta-analysis
What is the first stage of a systematic review?
Define question:
- based on a well defined PICO, is the question answerable?, is there already an up to date well conducted systematic review?
What is the second stage of a systematic review?
Write protocol
- why is the review important?
- description of the methods to be used: search strategy, data collection and analysis
- ensures methods and problems are considered beforehand
- Reduces the introduction of bias - prevents cheating as the protocol has to be registered
What is the third stage of a systematic review?
Search for evidence/studies
- Comprehensive identification of relevant studies using: bibliographic databases (medline, embase), grey literature, non-english literature, contact key experts/authors
A good review should look at all languages
What is the fourth stage of a systematic review?
Select relevant studies from search
- unbiased selection of studies for review
- 2 reviewers independently using pre-defined criteria based on PICO to select relevant studies
- reduce errors and bias - looking at very specific criteria to include/exclude - first you do abstracts then you do full paper
What is the fifth stage of a systematic review?
Appraise studies
- unbiased assessment of methodological quality of each study
- two reviews independently using checklist e.g. CASP
- reduce errors and bias
Cochrane database have a manual of how to do a systematic review
What is the sixth stage of a systematic review?
Extract, analyse and summarise data
- unbiased extraction of relevant data from studies in review
- 2 reviewers independently using pre-defined approach to identify relevant data in studies
- pre-defined analysis of data
- use of validated appropriate methods of numerical data synthesis (meta-analysis)
What is the seventh stage of a systematic review?
Interpret review results
- unbiased interpretation
- considers the results in the context of features and quality of the studies included in the review
- eg. age of trial may mean the comparator out of date
What is the final stage of a systematic review?
Discussion and conclusions
- consider implications of review findings in the context of current practice and knowledge and outlines specific future research
Summary of findings
In a forest plot, what does the size of the square mean?
Relates to the weight given to the study in the review - very affect by SD = dispersion of results within the study
In a forest plot, what does the middle of the square mean?
represents the result e.g. the relative risk for a study
In a forest plot, what does the diamond mean?
Summarised result - the width of the diamond represents 95% CI
In a forest plot, what does the line in each study mean?
the line represents the 95% CI
In a forest plot, what does the mid line mean?
line of no difference between the trial groups
What is the I2 used for ?
measuring level of heterogeneity = any variability between studies in a systematic review- could be clinical, methodological, or statistical heterogeneity
<25 = level of difference that doesn’t matter
>75 = worry trials are too different
What papers are more likely to be published?
Positive results bias - authors are more likely to submit and editors are more likely to accept positive results than negative or inconclusive results
What is outcome reporting bias?
several outcomes in a trial are reported selectively depending on the strength and direction of those results
What is language bias?
Positive results and large studies are more likely to be in english
Negative results and smaller studies are less likely to be in english
Sometimes a time lag for non-english studies
How can study size influence bias?
Small studies most likely to find small non-significant effects
Small studies that find a large effect size (by chance) are more likely to be published
Small studies with non-significant results are less likely to be published
What are funnel plots?
Scatter plot of the treatment effect size against a measure of each study’s size or precision
- effect size = horizontal
- study size = vertical
Subjective assessment of symmetry
