Exam Two - Systematic reviews/Meta Analyses Flashcards
Narrative or systematic review?
general or focused topic
narrative
Narrative or systematic review?
no specific criteria required of studied included in review.
narrative
Narrative or systematic review?
search and selection methods not reported
narrative
Narrative or systematic review?
quality and/or level of evidence of studies included in review is rarely considered
narrative
Narrative or systematic review?
focused question PICO
systematic
Narrative or systematic review?
summary of findings from studies that meet specific criteria
systematic
Narrative or systematic review?
rigorous search and selection methods established a priori
systematic
Narrative or systematic review?
extensive analysis of the quality of individual studies included in review (usually)
systematic
why are systematic reviews helpful?
1 - it is nearly impossible to stay current
2 - SR reduce the tendency to read articles that agree with you or that you are looking for
what is step one of a SR?
state your PICO
PICO choices tend to become your…
inclusion and exclusion criteria
what are steps 2,3 of SR?
Define the lit search
- type of studies to be included
- patient population that will be included
- definition of “intervention”
what kind of studies are typically included in SR
RCTs
step 4 of a SR
comprehensive, objective literature search
- include database and search term provided
- beware of publication bias
- beware of self reporting bias
What is publication bias?
- tendency to only publish articles if you find a significant result
- overtime this results in an overabundance of articles showing a significant difference
How do you combat publication bias?
“grey literature” search for articles not available through database searching
- contact authors for unpublished data
- government studies
- dissertations/theses
what is selective reporting bias?
- outcomes that are measured in the study but not reported because another outcome had more significant results
- statistically significant outcomes are more likely to be chosen by authors for reporting
what is step 5 of SR
- retrieve/screen articles
- some do not meet predetermined criteria
- ex. must use a specific outcome measure
What is a pro of requiring a specific outcome measure reported to be included in SR
- facilitates homogeneity of the results and combining results between studies
What is a con of requiring a specific outcome measure reported to be included in SR
- excludes findings from RCTs that may affect the validity of the review’s conclusions
what is step 6,7 of SR
Review articles for quality/internal validity
- each article is typically reviewed by at least 2 reviewers
Why do you want each article to be reviewed by 2 reviewers to be included in your SR?
- assure the reader “garbage in/garbage out” does not apply
- occasionally used to statistically place more emphasis on results of high quality studies (weighting)
what is step 8,9 of the SR
synthesis/analyze findings
- the goal of SR is to synthesize results of all relevant studies that evaluate a common question to get an answer
What are the 2 methods of synthesizing results
1 - systematic review (qualitatively/descriptively)
2 - meta-analysis (statistically) we want to be able to do this one
the _________ of the study results typically determines if the data will be synthesized statistically
heterogeneity
What are two major things that contribute to the heterogeneity in design
1 - sample characteristics
2 - methodology
How can you tell if effects are reasonable similar between studies?
1 - degree of overlap of study confidence intervals
- “eyeball test”
- little overlap = little heterogeneity
2 - cochran Q or Chi Square tests
- significance = heterogeneity
- may not be reported if <5 studies included in the review
Can you do a systematic review or meta analysis if the results are heterogeneous?
NO! Results must be homogenous
What is step 10 of a SR?
report results quantitatively
- typically described as “minimal” “moderate” or “strong” evidence for or against an intervention (considers count and quality of the articles)
Can you have more than one comparison per systematic review?
yes, absolutely! Piccccoooooooo
What are the advantages for doing a meta-analysis? (5)
1 - increase power
2 - more precise estimate of effect (larger n)
3 - quantifies the magnitude of the effect
4 - resolve conflicting results
5 - may improve external validity
What is a big problem when trying to do a meta-analysis?
studies often use different outcome measures to quantify the same construct
What is the main solution for the big problem we’ve discussed with meta analysis
authors of meta analyses can transform the results of each study into a dimensionless measure (so no units to worry about)
What are some dimensionless measures for interval or ratio outcome measures?
standard mean difference
weighted mean difference
What are some dimensionless measures for nominal or ordinal outcome measures?
- relative risk
- odds ratio
Why is it good to use weighted mean difference?
places greater emphasis on studies with more subjects and less variability (greater confidence)
True or false, better quality articles (higher PEDro score) could have greater weight regardless of sample size?
true!
What should you know to be able to read forest plots of continuous DVs?
- size of each square corresponds to the weight of the study
- center of diamond is the average effect for all studies
- tips of diamond represents CI
IF THE TIP OF DIAMOND CROSSES 0.0 THE EFFECT OF THE INTERVENTION IS NOT SIGNIFICANT
if the tip of the diamond crosses ________- for continuous DV forest plots, then the effect of the intervention is not significant
zero
What should you know to be able to read forest plots of discerete DVs?
IF THE DIAMOND CROSSES THE 1.0 THE INTERVENTION DOES NOT SIGNIFICANTLY INFLUENCE THE LIKELIHOOD OF EXPERIENCING THE OUTCOME
What is a common SR result report?
- inconclusive results
Why are inconclusive SR results useful??
- clinicians can be misled by the results of ONE study that “agrees” with their philosophy
- an inconclusive or negative SR may lead to review of their practice patterns
Why is PRISMA useful?
Its like the PEDro for SR and MA
What type of questions should you ask when trying to discover if a SR is VALID?
1 - did the review address a focused question
2 - where the inclusion/exclusion criteria appropriate
3 - is it unlikely that important, relevant studies were missed
4 - was the validity of the studies included appraised
5 - were the reviewers blinded and free of bias
6 - were the results reasonable homogenous?
What type of questions should you ask when trying to determine the clinical utility of a SR?
1 - were the results clearly stated
2 - effect size or NNT calculated
3 - can the results be applied to my patients and setting
4 - were the important outcome measures (to you) included in the review
5 - will my patient consider the benefits of the intervention to outweigh the risk (and cost)?
