Systematic Review Flashcards
What is a systematic review?
A summary of (all available) evidence on a particular topic based on a clearly formulated question that uses systematic and explicit predefined methods to identify, select and critically appraise relevant research and to collect and analyse data using validated methods from the studies that are included in the review
What are some examples of statistical methods used to analyse systematic review results?
Meta-analysis
Narrative
Realist review
Meta-synthesis
Why conduct a systematic review?
- To help identify which forms of health-care work are helpful, ones that are not and that are harmful
- Methods used inc. steps to minimize bias in all parts of the process
- Health-care providers/policy makers need to be evidence based making judicious use of evidence and ensuring practice is up to date
- Busy health care providers need quick and effective ways of identifying what works best for their patient
What should a stringent study include?
Inclusion and exclusion criteria
What are the 3 key functions of a systematic review?
- Synthesise the evidence i.e. bring results from similar randomized trials together
- Understand heterogeneity in treatment effect
- Set the research and practice agenda
What does the term systematic refer to in a systematic review?
Combining the trials in a reliable way i.e. systematically
When would we be likely to use systematic reviews?
Systematic reviews are the bedrock of clinical guidance e.g. NICE guidance so reading and using them is one of the best ways to stay abreast of what is new in your field of interest
What is the hierarchy of evidence from least to most rigour?
- Editorials and expert opinion
- Case series and reports
- Case-control studies
- Cohort studies
- RCTs
- Systematic reviews
What are the types of quantitative review evidence?
- Prevention/treatment/therapy
- Causation
- Diagnosis/prognosis
- Cost/cost-effectiveness
- Prevalence
What are the types of qualitative reviews (meta-ethnographies)?
Experience/barriers/facilitators/meaning
Before you start a systematic review, what do you need to know?
Purpose of review Nature of research Q Population Team factors Search strategy Type of studies/data inc. Outcomes (1mary/2ndary) Methods of analysis/evaluation of outcome Conclusions and added value
How can you scope the literature to determine thee rationale for the systematic review?
- Cochrane library, PubMed, Google
- PROSPERO database to register it
- Has a similar review been done? If so, why undertake a new one?
- Awareness of possible nature and volume of evidence/implications for timeline and resource
- Further project ideas (evidence gaps highlighted in NICE guidelines)
What would you do when getting started?
Familiarise yourself with process and start preparing protocol following reporting guidelines (PRISMA-P) and consult standard reference books e.g. Cochrane handbook
How can you translate a research question into a review?
Using suitable frameworks:
- PICOS (quantitative) = Population, Intervention, Comparison, Outcome and Study design
- SPICE (qualitative) = Setting, Perspective/Perspective, Intervention, Comparison and Evaluation
BUT each one may not be applicable for every question/review
What should the protocol include?
- Informed by and refined from research questions
2. Inc. all relevant items (as stated by PRISMA and module guidelines)
What is GRADE?
Systematic approach to rating the certainty of evidence in systematic reviews
What relevant items should be included in a systematic review protocol?
- Search strategy (sources, terms, limits)
- Inclusion/exclusion criteria
- Data extraction
- Risk of bias assessment
- Analysis/synthesis plan (quantitative/qualitative)
- Presentation + interpretation (GRADE)
What should you do when literature searching and retrieving studies?
- Seek assistance from librarian
- Identify where you plan to search e.g. appropriate data bases
- Identify search terms
- Useful search techniques e.g. Free Test searching, Thesaurus or controlled language screening and combining terms (AND, OR, NOT)
What are the steps that take place after a search has been conducted?
- Identification: state no. of records identified through database searching and through other sources
- Screening: delete duplicates and state no. of records screened (inc. vs exc.)
- Eligibility: state no. of records assessed for eligibility (in/out) and no. of records excluded with reasons (PICOS/SPICE)
- Included studies: state no. of records included in qualitative/quantitative synthesis
How can you present results of a search? Why?
PRISMA flow chart/diagram that includes no. of hits, no. excluded and reasons for, the amount included in the review and the no. meta-analysed - shows transparency of search methods
What are the steps used after you have found studies to include in your review?
- Data abstraction: pre-defined data abstraction form linked to research Qs and aims
- Quality assessment using suitable tool to inform analysis and interpretation - GRADE (inc. high, low or unclear risk of bias)
- Analysis of results (quantitative/qualitative)
- Critical summary and write-up: present and interpret findings
Define selection bias.
Systematic bias in the way participants are selected
Define performance bias.
Systematic differences in the care provided to the participants in the comparison groups other than the intervention under investigation
Define detection bias.
Systematic differences between comparison groups in outcome assessment (blinding)
Define attrition bias.
Systematic differences between comparison groups in the loss of participants from the study (drop-outs)
Define reporting bias.
Selective reporting of study outcomes
What other types of bias exist?
Funding (conflict of interest if funder intervened)
Error in results
What is a meta-analysis? What is the downside to using these?
If the results of the individual studies are combined produce an overall statistic requiring statistical packages to conduct the analysis (STATA + RevMan) HOWEVER, it will be subject to any biases that arise from study selection process and may produce a mathematically precise but clinically misleading result (may want to remove studies at high risk of bias)
What information has to be harmonized to conduct a meta-analysis?
Outcome being examined Timeframe for outcome Summary effect size used No. of trials included Heterogeneity in comparison
What is heterogeneity?
The % of variation between studies in this meta-analysis due to differences between studies and not just sampling error (I^2) which shapes our understanding of pooled effect e.g. is this the true effect or the average effect in a distribution?
What are the different classes of heterogeneity?
0% = none 25% = low 50% = moderate 75% = high
How can you present the results of your meta-analysis?
Forest plot
What is the final row and diamond of the forest plot? What happens if it crosses the vertical line 1 or lies on either side?
Pooled effect (row) and point estimate (diamond) of all studies analysed
If it crosses the line at 1, there is no statistically significant difference however if it lies towards the side favouring the intervention, there is a statistically significant result favouring intervention and vice versa
What is the middle vertical line in a forest plot?
Line of nil effect where there is no statistically significant difference (null hypothesis)
For OR = 1
For mean difference = 0
How can you reduce heterogeneity?
Conduct further analysis
Look at factors alone to see if this reduces it
How do you analyse results of a systematic review that are not quantitative and therefore cannot be meta-analysed?
Narrative analysis: description of studies that cannot go into a statistical meta-analysis as they are different and difficult to harmonize as well as being qualitative
What types of bias exist in a systematic review?
Publication Duplicate (multiple) publication Language Outcome reporting Citation
What is duplicate (multiple) publication bias?
If a systematic review counts the same study more than once as it has been published > 1 time exaggerating the outcome
What it outcome reporting bias?
Reporting of outcomes that deviate from set plan because they encountered problems
What is citation bias?
Researchers do each other a favour by citing each other a lot
How can you represent publication bias if there is 10 or more studies? What does each dot represent? What should it look like?
Funnel plot where each dot represents a single study - should look like an inverted V symmetrical is publication bias is NOT present (will be asymmetrical and skewed if this type of bias exists)
What is publication bias?
Studies with positive results are more likely to be published than negative studies - the subsequent over-representation of positive studies in systematic reviews may mean that reviews are biased toward a positive result.
What should be included in your review team?
- 2nd reviewer
- Reference management, statistical and qualitative analysis software language support and resources
- Advice on literature search, review methodology and content expertise
What should you consider in terms of other planning issues?
- Iterative process: seek feedback early
- Consider protocol registration (PROSPERO)
- Plan your timeline - do NOT underestimate time required (3 months - 4 years)
How can you borrow wisdom from previous work?
- Find systematic review of similar nature
- Cochrane library (Medicine and Health-Related)
- Campbell library (Education, Social Welfare, International Development etc.)
- Published reviews and protocols in PubMed/Medline and Systematic Reviews journal
What are the contents of a high quality systematic review?
Search strategy inclusive and reproducible
Flow of all studies/papers (PRISMA)
How inc. and exc. criteria apply (PICOS/SPICE)
Table of characteristics of studies (inc. and exc. reasons)
Quality assessment cohesive synthesis/analysis (+/- meta-analysis)
Practical application (context) of findings
Strengths and limitations of systematic review
What are the 3 main sections in the CASP checklist?
- Are the results of the study valid?
- What are the results?
- Will the results help locally?
What do the squares in a forest plot show?
The % weight given to each study indicating proportion that this study’s results contributed to pooled result in meta-analysis- studies weighted by inverse of variance for estimated measure of effect i.e. more precise studies with narrower CIs have a higher weighting
What are the horizontal lines sticking out from squares on a forest plot?? Why are some longer?
Confidence intervals - less precise studies have bigger CIs and longer lines relating not to study size but the variability of patient response
What would a odds ratio (OR) of 1.42with 95% confidence interval (CI) 1.17-1.73 indicate?
The odds are 42% higher
It lies between the 95% CIs and the CI does not inc. 1 (null hypothesis) meaning the null hypothesis can be rejected at the 5% level OR result is statistically significant
