Exam Two - Systematic reviews/Meta Analyses

Question 1

Narrative or systematic review?
general or focused topic

Answer 1

narrative

Question 2

Narrative or systematic review?
no specific criteria required of studied included in review.

Answer 2

narrative

Question 3

Narrative or systematic review?
search and selection methods not reported

Answer 3

narrative

Question 4

Narrative or systematic review?
quality and/or level of evidence of studies included in review is rarely considered

Answer 4

narrative

Question 5

Narrative or systematic review?
focused question PICO

Answer 5

systematic

Question 6

Narrative or systematic review?
summary of findings from studies that meet specific criteria

Answer 6

systematic

Question 7

Narrative or systematic review?
rigorous search and selection methods established a priori

Answer 7

systematic

Question 8

Narrative or systematic review?
extensive analysis of the quality of individual studies included in review (usually)

Answer 8

systematic

Question 9

why are systematic reviews helpful?

Answer 9

1 - it is nearly impossible to stay current
2 - SR reduce the tendency to read articles that agree with you or that you are looking for

Question 10

what is step one of a SR?

Answer 10

state your PICO

Question 11

PICO choices tend to become your…

Answer 11

inclusion and exclusion criteria

Question 12

what are steps 2,3 of SR?

Answer 12

Define the lit search
- type of studies to be included
- patient population that will be included
- definition of “intervention”

Question 13

what kind of studies are typically included in SR

Answer 13

RCTs

Question 14

step 4 of a SR

Answer 14

comprehensive, objective literature search
- include database and search term provided
- beware of publication bias
- beware of self reporting bias

Question 15

What is publication bias?

Answer 15

• tendency to only publish articles if you find a significant result
• overtime this results in an overabundance of articles showing a significant difference

Question 16

How do you combat publication bias?

Answer 16

“grey literature” search for articles not available through database searching
- contact authors for unpublished data
- government studies
- dissertations/theses

Question 17

what is selective reporting bias?

Answer 17

• outcomes that are measured in the study but not reported because another outcome had more significant results
• statistically significant outcomes are more likely to be chosen by authors for reporting

Question 18

what is step 5 of SR

Answer 18

• retrieve/screen articles
• some do not meet predetermined criteria
• ex. must use a specific outcome measure

Question 19

What is a pro of requiring a specific outcome measure reported to be included in SR

Answer 19

• facilitates homogeneity of the results and combining results between studies

Question 20

What is a con of requiring a specific outcome measure reported to be included in SR

Answer 20

• excludes findings from RCTs that may affect the validity of the review’s conclusions

Question 21

what is step 6,7 of SR

Answer 21

Review articles for quality/internal validity
- each article is typically reviewed by at least 2 reviewers

Question 22

Why do you want each article to be reviewed by 2 reviewers to be included in your SR?

Answer 22

• assure the reader “garbage in/garbage out” does not apply
• occasionally used to statistically place more emphasis on results of high quality studies (weighting)

Question 23

what is step 8,9 of the SR

Answer 23

synthesis/analyze findings
- the goal of SR is to synthesize results of all relevant studies that evaluate a common question to get an answer

Question 24

What are the 2 methods of synthesizing results

Answer 24

1 - systematic review (qualitatively/descriptively)
2 - meta-analysis (statistically) we want to be able to do this one

Question 25

the _________ of the study results typically determines if the data will be synthesized statistically

Answer 25

heterogeneity

Question 26

What are two major things that contribute to the heterogeneity in design

Answer 26

1 - sample characteristics
2 - methodology

Question 27

How can you tell if effects are reasonable similar between studies?

Answer 27

1 - degree of overlap of study confidence intervals
- “eyeball test”
- little overlap = little heterogeneity
2 - cochran Q or Chi Square tests
- significance = heterogeneity
- may not be reported if <5 studies included in the review

Question 28

Can you do a systematic review or meta analysis if the results are heterogeneous?

Answer 28

NO! Results must be homogenous

Question 29

What is step 10 of a SR?

Answer 29

report results quantitatively
- typically described as “minimal” “moderate” or “strong” evidence for or against an intervention (considers count and quality of the articles)

Question 30

Can you have more than one comparison per systematic review?

Answer 30

yes, absolutely! Piccccoooooooo

Question 31

What are the advantages for doing a meta-analysis? (5)

Answer 31

1 - increase power
2 - more precise estimate of effect (larger n)
3 - quantifies the magnitude of the effect
4 - resolve conflicting results
5 - may improve external validity

Question 32

What is a big problem when trying to do a meta-analysis?

Answer 32

studies often use different outcome measures to quantify the same construct

Question 33

What is the main solution for the big problem we’ve discussed with meta analysis

Answer 33

authors of meta analyses can transform the results of each study into a dimensionless measure (so no units to worry about)

Question 34

What are some dimensionless measures for interval or ratio outcome measures?

Answer 34

standard mean difference
weighted mean difference

Question 35

What are some dimensionless measures for nominal or ordinal outcome measures?

Answer 35

• relative risk
• odds ratio

Question 36

Why is it good to use weighted mean difference?

Answer 36

places greater emphasis on studies with more subjects and less variability (greater confidence)

Question 37

True or false, better quality articles (higher PEDro score) could have greater weight regardless of sample size?

Answer 37

true!

Question 38

What should you know to be able to read forest plots of continuous DVs?

Answer 38

• size of each square corresponds to the weight of the study
• center of diamond is the average effect for all studies
• tips of diamond represents CI
  IF THE TIP OF DIAMOND CROSSES 0.0 THE EFFECT OF THE INTERVENTION IS NOT SIGNIFICANT

Question 39

if the tip of the diamond crosses ________- for continuous DV forest plots, then the effect of the intervention is not significant

Answer 39

zero

Question 40

What should you know to be able to read forest plots of discerete DVs?

Answer 40

IF THE DIAMOND CROSSES THE 1.0 THE INTERVENTION DOES NOT SIGNIFICANTLY INFLUENCE THE LIKELIHOOD OF EXPERIENCING THE OUTCOME

Question 41

What is a common SR result report?

Answer 41

• inconclusive results

Question 42

Why are inconclusive SR results useful??

Answer 42

• clinicians can be misled by the results of ONE study that “agrees” with their philosophy
• an inconclusive or negative SR may lead to review of their practice patterns

Question 43

Why is PRISMA useful?

Answer 43

Its like the PEDro for SR and MA

Question 44

What type of questions should you ask when trying to discover if a SR is VALID?

Answer 44

1 - did the review address a focused question
2 - where the inclusion/exclusion criteria appropriate
3 - is it unlikely that important, relevant studies were missed
4 - was the validity of the studies included appraised
5 - were the reviewers blinded and free of bias
6 - were the results reasonable homogenous?

Question 45

What type of questions should you ask when trying to determine the clinical utility of a SR?

Answer 45

1 - were the results clearly stated
2 - effect size or NNT calculated
3 - can the results be applied to my patients and setting
4 - were the important outcome measures (to you) included in the review
5 - will my patient consider the benefits of the intervention to outweigh the risk (and cost)?