Clinical epidemiology: Systematic reviews of RCTs Flashcards
What are some reasons why there are differences between RCTs?
Differences:
- in people compared
- in care, other than that being investigated
- in the way treatment effects are assessed
- in follow up
Available evidence:
- differences in reporting
- biased selection
Also obsolescence over time, changes in risk profile of patients, missing information
Hopefully not:
- bad science
- fraudulent science
What is a systematic review?
Review of all literature on one particular topic using scientific methods:
- clear question being addressed - PICO
- explicit and detailed statement of the methods used
- comprehensive ascertainment of the literature relevant to the review question
- steps to reduce bias in the review process with or without: a statistical technique used to produce a numerical summary of the size of effect with confidence intervals
What are the advantages of systematic reviews?
- can be the best source of evidence on the effects and effectiveness of healthcare interventions
- assimilate large amounts of research evidence
- provide reliable unbiased estimates of effect
- increase precision of estimates of effects
- provide information about the generalisability and consistency of effects
- Identify what information is missing
- Useful for making decisions
How does the purpose of a narrative review (non-systematic review) differ from a systematic review?
Narrative review:
- background information and current thinking
Systematic review:
- identify, evaluate and summarise current evidence
How do the methods of a narrative review (non-systematic review) differ from a systematic review?
Narrative review:
- Not specified
Systematic review:
- specified and reproducible
How does the review question of a narrative review (non-systematic review) differ from a systematic review?
Narrative review:
- often general discussion with no stated hypothesis
Systematic review:
- starts with clear question or hypothesis to be tested
How does the identification of studies of a narrative review (non-systematic review) differ from a systematic review?
Narrative review:
- well known articles, filling cabinet or selected e.g. medline only
Systematic review:
- planned search strategy for published and unpublished studies
How does the study selection for inclusion of a narrative review (non-systematic review) differ from a systematic review?
Narrative review:
- do not usually describe why studies are included/excluded
Systematic review:
- explicit criteria for included and excluded studies
How does the quality of assessment and data extraction of a narrative review (non-systematic review) differ from a systematic review?
Narrative review:
- subjective data extraction, do not usually consider study methods and bias
Systematic review:
- objective extraction of data, critical examination of study methods and bias -standard quality assessment method
How does the analysis of results of a narrative review (non-systematic review) differ from a systematic review?
Narrative review:
- various; often don’t consider differences between studies
Systematic review:
- Validated methods, assess heterogeneity, meta-analysis
What is the first stage of a systematic review?
Define question:
- based on a well defined PICO, is the question answerable?, is there already an up to date well conducted systematic review?
What is the second stage of a systematic review?
Write protocol
- why is the review important?
- description of the methods to be used: search strategy, data collection and analysis
- ensures methods and problems are considered beforehand
- Reduces the introduction of bias - prevents cheating as the protocol has to be registered
What is the third stage of a systematic review?
Search for evidence/studies
- Comprehensive identification of relevant studies using: bibliographic databases (medline, embase), grey literature, non-english literature, contact key experts/authors
A good review should look at all languages
What is the fourth stage of a systematic review?
Select relevant studies from search
- unbiased selection of studies for review
- 2 reviewers independently using pre-defined criteria based on PICO to select relevant studies
- reduce errors and bias - looking at very specific criteria to include/exclude - first you do abstracts then you do full paper
What is the fifth stage of a systematic review?
Appraise studies
- unbiased assessment of methodological quality of each study
- two reviews independently using checklist e.g. CASP
- reduce errors and bias
Cochrane database have a manual of how to do a systematic review
What is the sixth stage of a systematic review?
Extract, analyse and summarise data
- unbiased extraction of relevant data from studies in review
- 2 reviewers independently using pre-defined approach to identify relevant data in studies
- pre-defined analysis of data
- use of validated appropriate methods of numerical data synthesis (meta-analysis)
What is the seventh stage of a systematic review?
Interpret review results
- unbiased interpretation
- considers the results in the context of features and quality of the studies included in the review
- eg. age of trial may mean the comparator out of date
What is the final stage of a systematic review?
Discussion and conclusions
- consider implications of review findings in the context of current practice and knowledge and outlines specific future research
Summary of findings
In a forest plot, what does the size of the square mean?
Relates to the weight given to the study in the review - very affect by SD = dispersion of results within the study
In a forest plot, what does the middle of the square mean?
represents the result e.g. the relative risk for a study
In a forest plot, what does the diamond mean?
Summarised result - the width of the diamond represents 95% CI
In a forest plot, what does the line in each study mean?
the line represents the 95% CI
In a forest plot, what does the mid line mean?
line of no difference between the trial groups
What is the I2 used for ?
measuring level of heterogeneity = any variability between studies in a systematic review- could be clinical, methodological, or statistical heterogeneity
<25 = level of difference that doesn’t matter
>75 = worry trials are too different
