Systematic Reviews Flashcards
ideal world vs. real world for studies
ideal world:
- we have a clinical Q
- we can easily find an access to the perfect study that answers out exact question in our population of interest
- this study is to limited by bias or confounding
- this study is very large and has a precise estimate of the measure of association (the 95% CI’s are narrow), and the estimate resembles ‘the truth’
real world:
- we have a clinical question
- we look for evidence (conduct a review)
- we fins a lot of papers that answer our question to some extent
- all are limited by some degree by: bias, confounding, lack of precision and/or generalisability
- we don’t know what ‘the truth’ is
someone may have done a review that answers out question. but how do we know whether to trust the review?
Two mains types of review:
Narrative: may be heavily influenced by opinion (even if they are written by experts)
- question is often broad in scope
- sources and search are not usually specified, potentially biased
- selection is not usually specified, potentially biased
- appraisal can be variable
- synthesis is often a qualitative summary
- inferences are sometimes evidence based
Systematic: reliable, transparent and systematic:
- question is often a focused clinical question
- the sources and each are comprehensive sources and explicit search strategy
- selection is criterion-based selection, uniformly appraised
- appraisal is rigorous critical appraisal
- synthesis is quantitative summary
- inferences are usually evidence-based
describe the systematic reviews effectiveness
- systematic review is a research methodology
- number of systematic reviews done/used since 2000 has shot up exponentially
- there are research methods characteristic of systematic review and guidelines for reporting
- there is a register of proposed systematic reviews
- is is possible to do systematic review to address different questions, such as: effectiveness, prognosis, diagnostic test accuracy, prevalence and incidence, patient experiences
Research study design effectiveness:
1. systematic reviews
2. RCTS
3. Cohort studies
4. Case-control studies
5. case series, case reports
6. editorials, expert opinions
what is Cochrane library?
a website that has published protocols and guidelines on how to do systematic reviews and they do many themselves
- global
- independent
- international and interdisciplinary network
- not for profit
- no commercial sponsorship
- freely available to everyone in NZ
why are systematic reviews done?
- collate evidence and synthesise their results (come up with a summary of what the literature shows)
- done well, systematic review methods reduce bias that may otherwise be encountered wit narrative reviews
how are systematic reviews conducted?
- formulation of a clear question
- write a protocol for the review
- search for relevant studies
- collect data from studies
- assessment of included studies
- synthesis of findings
- interpretation of results
what could a protocol include?
- question
- must have all aspects of PECOT framework
- need to pre-specify time period before review - relevance and 3. objectives
- why is it important to do this review?
- objective statement (of intent) - research strategy
- needs to be replicable and transparent
- involves conducting a search of the literature, which needs tone very specific and search in specific databases (need to outline what databases were looked at) - selection criteria and 6. eligibility screen
- including and exclusion criteria of studies - risk of bias
- a way to assess the quality of the studies you are looking at
- can use a risk of bias chart, which includes the studies down the left and the potential sources of bias along the top (green is low risk of bias, red is high risk of bias, orange is unspecified) - data extraction and 9. data synthesis
- MAY involve meta-analysis
- extract and review data for your research Q, synthesise so it can be looked at easily/properly
what would a flow diagram be used for in a systematic review?
- common one is a prism flow diagram
- shows the process of what you are doing (consists of identification, screening, eligibility, included)
what is meta analysis?
If the results of the individual studies are combined to produce an overall statistic, this is called meta-analysis. They measure the benefits and harms but collecting data from more than one trial, and combining them to generate an average result. This aims to provide a more precise estimate of the effects of an intervention and to reduce uncertainty
- see a similar process as when you take lots of samples and find an average
- heterogeneity: when something varies across different groups it is heterogeneous
- homogeneity: when something is consistent across different groups it is homogeneous
describe how you interpret forrest plots (graphical presentation of results)
- list of studies on the lest, each of them is listed by the name of the first author or the company, as well as date of publication
- on the top left we can see the two things we are comparing, with the number of events and people in each arm listed downwards
- the measure of association (relative risk/risk ratio) is on the top right, with the results of the studies and the 95% CI listed for each study
- on the very right there are he 95% CI’s drawn out and the dot with is the point estimate. Vertical line is the null value (it is 1 in this case)
- on the bottom right, it is an explanation of what each side of the line means and the scale for the RR
- have weight along the top too, which is the meaningfulness/power of the study. also represented by the size of the point estimate box
- diamond show the pooled result, line through it is the pooled risk ratio, using the weights listed in the column. width of diamond shows the CI for the pooled results
- bottom left is statistical analysis which says the extant to which the studies are heterogeneous. the I squared gives this as a percentage.
describe what you are looking for and what the limitations are for interpretation
- principle findings
- limitations to evidence (risk of bias, publication bias)
- implications for practice and research
“rubbish in, rubbish out” - if you are using bad studies, then your collected data will not be of much use/not be very accurate
what are the challenges and advantages of systematic reviews
challenges:
- doing all steps well
- publication bias
- poor quality trials/studies
- heterogeneity
- can lead to: conflicting results and/or inconclusive results
advantages:
- redpoducability, rigour
- comprehensive
- transparent limits
- gaps in knowledge
- basis for decisions
describe an ideal world with the use of Cochrane
- we have a clinical Q
- we fins a Cochrane systematic review and mata-analysis that answers our Q almost perfectly
- the review includes studies that have low risk of bias
- the meta-analysis shows a pooled estimate with a narrow 95% CI
- we can apply the results to our clinical Q
