L05 - AAAA systematic reviews Flashcards
What does ‘assess’ in the AAAA framework constitute?
- Formulate a clear clinical question (PICO framework)
- Identify the study design that would provide the most accurate answer
What does ‘access’ in the AAAA framework constitute?
- Search for and retrieve the research
What does ‘identify’ in the AAAA framework constitute?
- Identify the strengths and weaknesses of the research you find and understand what the results mean
What does ‘act’ in the AAAA framework constitute?
- Apply the evidence, taking into account its limitations, alongside other information (e.g. patient pref)
What is positive results bias?
When authors are more likely to submit, or editors accept, positive than negative or inconclusive results
What is outcome reporting bias?
When several outcomes within a trial are measured but are reported selectively depending on the strength and direction of those results
What is the PICO framework?
P = Patient popn - how describe group of patients I = Intervention or issue C = comparator - main alt intervention (comparison intervention) O = Outcomes - context
Advantages of systematic reviews?
- Can be best source of evidence on effects and effectiveness of healthcare interventions
- Assimilation of large amounts of research evidence
- Provide reliable unbiased estimates of effect
- Inc precision of estimates of effects
- Proc inf about the generalisability and consistency of effect
- Identify what info is missing
- Useful for making decisions
What is evidence based medicine?
- Mix of indiv clinical expertise, best external evidence and patient values/ expectations
- Optimise dec making by using best available research
What are some reasons for differences between RCT’s?
- Chance
- Reporting of available evidence
- Selection from available evidence
Differences in: - Ppl
- Intervention/ comparator
- Care other than what being investigated
- Assessment of treatment effects
- Follow-up
What is publication bias?
The outcome of an experiment or research study influences the decision whether to publish or otherwise distribute it.
What are the different types of publication bias?
- +VE results bias
- Outcome reporting bias
What is positive results bias?
When authors more likely to submit or editors accept, positive than negative or inconclusive results
What is outcome reporting bias?
When several outcomes within a trial are measured but are reported selectively depending on the strength and direction of those results
What are the benefits of reviews?
- Vol of lit condensed
- New info made accessible
- Relevant info
- Saves time?
What is a systematic review?
The research method used to synthesis multiple pieces of research evidence on the effectiveness of a treatment
- Review of all lit
- On one particular topic
- Using scientific methods
List the hierarchy or evidence (for effectiveness)
- Systematic reviews
- RCT’s
- Cohort studies
- Case control studies
- Case series and reports
- Expert opinion ‘anecdotal’ evidence
What are the stages in a systematic review?
- Define question (what/ why/ PICO)
- Write protocol - desc of methods
- Search for evidence/ studies - comprehensive identification of relevant studies
- Select relevant studies from search - unbiased selection of studies for review
- Appraise studies - unbiased assessment of methodological quality of each study
- Extract, analyse and summarise data - unbia extraction
- Interpret review results
- Discussion and conclusions
Why is writing a protocol an important stage of writing a systematic review?
- Desc of methods
- Ensures methods and problems are considered a priority
- Avoids introducing bias
How might you search f for evidence/ studies in a systematic review?
- Bibliographic databases
- MEDLINE, EMBASE
- Non-eng lang lit
- Grey lit hard to identify)
How would you select relevant studies from the search in a systematic review?
Two reviewers independently using pre-defined criteria (based on PICO) to select relevant studies from the search
- reduces errors
How would you appraise studies in a systematic review?
Two reviewers independetly using checklist (e.g. CASP - critical appraisal skills programme)
- reduces errors
How would you extract, analyse and summarise data in a systematic review?
Two reviewers independently using predefined approach to identify relevant data
- Predefined analysis of data
- Use of validated appropriate methods of numerical data synthesis (meta-analysis)
- Predefined exploration of possible effect modifiers
What is a meta-analysis?
A statistical analysis that combines the results of multiple scientific studies that address the same question (each study has some degree of error reported)
How is the result from a meta-analysis displayed?
Displayed as a forest plot
What does the square represent in a forest plot?
Middle of square = the result (the relative risk for a study)
What does the line represent in a forest plot?
Line = the confidence interval
What does the diamond represent in a forest plot?
Diamond = summarised results (meta-analysis)
Width of diamond = the CI
What does the size of the square represent in a forest plot?
Square size = weight given to the study in the review
How would you interpret review results in a systematic review?
Consider the results in the context of the quality of the studies included in the review
How would you discuss and conclude results in a systematic review?
Consider the implications of the review findings in the context of current practice and knowledge and outline specific future research
- Summary of findings
What are the advantages of systematic reviews?
- Can be best source of evidence on the effects and effectiveness of healthcare interventions
- Assimilation of large amounts of research evidence
- Reliable unbiased estimates of effect
- Inc precision of estimates of effects
- Provide info about the generalisability and consistency of effect
- Identify what info is missing
- Useful for making decisions
Where can you find systematic reviews of effectiveness in healthcare?
- Cochrane library
- Epistemonikos
- NHS evidence
- MEDLINE (and/ or EMBASE)
What is the 3 part process of CASP?
- Is the review trustworthy/ valid? (screening questions/ detailed questions)
- What were the review’s results?
- How will the result’s help? (relevance)
What are some crtical appraisal questions for systematic reviews?
- Did the review ask a clearly focused question
- In terms of PICO easily identifiable? - Did the review include the right type of study?
- Address the review’s questions?
- Appropriate study design? - Did the reviewers try to identify all relevant studies?
- Bibliographic databases?
- Follow up from ref lists?
- Personal contact with experts?
- Unpublished studies?
- Non-Eng lang studies? - How did the R decide which studies to include and did they assess the quality (validity) of the incl studies?
- Clear predetermined strat to determine which studies to incl in review
- How assess quality?
- Defined sel criteria based on PICO
- Defined validity process (CASP checklist)
- > one assessor - If the results of the studies comb tgt (meta-ana) was it reasonable to do so?
- Results of each study clearly displayed?
- Results similar from study to study? - How results presented and what is main result?
- How expressed results (relative risk, mean diff)
- Size of result and how meaningful
- Sum-up result in one sentence? - Precision of results?
- CI reported
- p-value reported where CI are unavailable - Can results be app to local popn?
- Can provide same I in your setting?
- Local setting differ from review?
- popn sample covered diff from your popn so that diff results prod? - Were all important outcomes considered?
Outcomes from POV of
- indiv
- policy makers and professionals
- Fam/ carers
- Wider community - Should policy or practice change as a result of evidence contained in this review?
- Benefit reported outweigh harm/ cost?
- Info not reported can it be filled in from elsewhere?
What is methodological heterogeneity?
Different study designs combined
What is clinical heterogeneity?
Different patient characteristics, settings and interventions
What is statistical heterogeneity?
Differences in reported effect size between the trials (Chi sqr test, I^2)
