QUIZ 3: Chapter 27 Systematic Reviews of Research Evidence Flashcards
What type of review does this describe:
“a review that methodically integrates research evidence about a specific research question using carefully developed sampling & data collection procedures that are spelled out in advanced in a protocol”
Systematic review (p. 653
True or False:
systematic reviews, unlike other types of reviews, are a process of developing, testing, & adhering to a protocol with explicit rules for gathering data - the research evidence - from studies that address a particular problem
True
Is a systematic review of evidence from qualitative studies also known as a meta-analysis?
No, a meta analysis is a systematic review of evidence from QUANTITATIVE studies
Yes or No:
Is the essence of a meta-analysis that information from various studies be used to develop a common metric, the effect size?
Yes
and the effect size averages are averaged across studies, which yields information about the EXISTENCE of a relationship between variables as well as the MAGNITUDE of this relationship
Name 3 advantages to meta-analyses?
(may be helpful to think of the chorus to that song back in the day…“Are you down with O.P.P?” as an acronym for remembering these advantages)…on another note, i had NO idea what that actually stood for until i just looked it up, whoa!
Objectivity
Power
Precision
Which advantage does this describe (objectivity, power, or precision):
- draws conclusions about how big an effect an intervention has
- estimate effect size across multiple studies yields smaller confidence intervals than individual studies
Precision (p. 654-655)
Which advantage does this describe (objectivity, power, or precision):
- makes decisions more explicit
- integration is statistical
Objectivity (p. 654)
Which advantage does this describe (objectivity, power, or precision):
- probability of detecting a true relationship between the IV and DV
- combining results across multiple studies increases this
Power (p. 654)
Which step is out of order within the “Steps in a Meta Analysis”:
- Formulating a problem
- Designing the meta-analysis study
- Searching literature for data
- Analyzing data
- Extracting/encoding data for analysis
- Calculating effects
- Evaluating study quality
- Writing report
4 and #7 should be switched
Which step of the steps in a meta analysis does this describe (formulating a problem, designing the meta-analysis study, searching literature for data, evaluating the study quality, extracting/encoding data for analysis, calculating effects, analyzing data, writing report):
- essential for collecting & integrating meaningful data
- start with broad question
- narrow down to specific question (key constructs conceptually defined; indicate boundaries of inquiry)
- develop a problem statement
Formulating a problem
Which step of the steps in a meta analysis does this describe (formulating a problem, designing the meta-analysis study, searching literature for data, evaluating the study quality, extracting/encoding data for analysis, calculating effects, analyzing data, writing report):
- using standardized search strategies
- deciding on published vs. grey literature
- addressing publication bias
Searching the literature
Which step of the steps in a meta analysis does this describe (formulating a problem, designing the meta-analysis study, searching literature for data, evaluating the study quality, extracting/encoding data for analysis, calculating effects, analyzing data, writing report):
- develop or select quality assessment scales
- decide on coding for quality elements
Evaluating the study quality
Which step of the steps in a meta analysis does this describe (formulating a problem, designing the meta-analysis study, searching literature for data, evaluating the study quality, extracting/encoding data for analysis, calculating effects, analyzing data, writing report):
- project organization
- planned sampling procedures
- quality of primary study
- statistical heterogeneity
Designing the meta-analysis study
Which step of the steps in a meta analysis does this describe (formulating a problem, designing the meta-analysis study, searching literature for data, evaluating the study quality, extracting/encoding data for analysis, calculating effects, analyzing data, writing report):
- calculate effect size for each study
- if outcomes across studies are on identical scales (subtract the mean of outcome from treatment and control groups)
- if outcomes are on different scales (use neutral index - Cohen d)
- if outcomes across studies are dichotomous (calculate relative risk, odds ratio, & absolute risk reduction)
Calculating effects
Which step of the steps in a meta analysis does this describe (formulating a problem, designing the meta-analysis study, searching literature for data, evaluating the study quality, extracting/encoding data for analysis, calculating effects, analyzing data, writing report):
- extract relevant info about variables of interest
- develop data extraction form
- develop codes & coding manual
Extracting/encoding data
Which step of the steps in a meta analysis does this describe (formulating a problem, designing the meta-analysis study, searching literature for data, evaluating the study quality, extracting/encoding data for analysis, calculating effects, analyzing data, writing report):
- disseminating findings
- Cochrane & PRISMA
writing report
Which step of the steps in a meta analysis does this describe (formulating a problem, designing the meta-analysis study, searching literature for data, evaluating the study quality, extracting/encoding data for analysis, calculating effects, analyzing data, writing report):
- calculate summary statistic that captures effect for each study
- pooled effect estimate is computed as weighted average for all studies
- identify heterogeneity (visual inspection with forest plots, statistical inspection with chi-square)
- decide on fixed effects vs. random effects
- examine factors affecting heterogeneity
- handle study quality
- address publication bias
analyzing data
What do these 2 factors affect (within analyzing the data)?:
- subgroup analyses (involves splitting the effect size info from studies into distinct categorical groups, such as gender)
- meta-regression (involves predicting the effect size based on possible explanatory factors)
heterogeneity (pgs. 663-664)
Can you name 4 strategies that deal with the issue of study quality in a meta-analysis?
- review inclusion/exclusion criteria
- determine if sensitive analysis is needed
- consider quality as basis for exploring heterogeneity
- weight studies according to quality criteria (p. 664)
I didn’t create a flashcard for this, but it may be helpful to look at the funnel plot figures in chapter 27 that examine the possibility of a publication bias
Page 665
