**PHEPB Systematic Review: What is evidence synthesis? Flashcards
Define a systematic review
- AKA overview, of the literature represents ‘a systematic assembly, critical appraisal and synthesis of all relevant studies on a specific topic’
- A systematic review can be quantitative (a ‘meta analysis’), or qualitative
What is the purpose of a systematic review?
- To summarise and simplify complex and vast literature
- Increase precision of effect estimates, and improve statistical power
- Assess the consistency of the evidence, and thereby the generalisability of the findings
- Establish possible explanations for any inconsistencies
- Highight areas of relative certainty or deficiencies in evidence
Describe the design of a systematic review
Define:
-primary exposure and outcome
-measures of exposure or intervention, outcome or effect
Exclusion and inclusion criteria for the systematic retreival of relevant studies
Detail the method of extracting the relevant quantitative/qualitative info
Summarise the evidence using the appropriate stats
Interpret the results acknowledging systematic error (bias) and random variation (chance)
What is health outcome? Choice and definition of health outcome depends on…
Choice and definition of health outcome depends upon the purpose of the review
Defined outcomes: mortality, birthweight etc
Range of outcomes: non-fatal illness, indices of clinical severity, symptom report, measures of health service usage, QoL indices
Choice of outcome often depends on and is limited by what is published and available
May need to be pragmatic in terms of finding different studies - defined vs range of outcomes
What is the difference between observational and experimental studies?
Exposures or interventions
* Observational studies
– Often different measures of exposure
– Self report vs. measured / observed exposure
– Information on past or cumulative exposure
– Type of exposure or critical periods of exposure
* Experimental studies
– Usually report similar types of intervention
– Often differences in…
* dosage schedule
* compliance or uptake
* type of intervention (drugs vs. diet)
* timing of intervention (late vs. early)
How can we systematically retirive studies?
- Searches of electronic databases (e.g. Medline,
Embase, Web of Science, Trial registers) using text words and subject headings (e.g., MESH) - Retrospective citations in recent reviews
- Prospective citations of key methodological or
classic papers (Science Citation Index) - Hand searches of relevant journals
- Increasing use of data from unpublished studies
– so called “grey literature”
What is grey literature?
Studies that are null do not tend to be published → grey literature (publishing grey literature would help balance affirmative findings from null findings)
A widely accepted definition for grey literature is. “info produced on all levels of government, academia, business and industry in electronic and print formats not controlled by commercial publishing” ie. where publishing is not the primary activity of the producing body
When you conduct a systematic review, what will you always follow?
This framework. Note RCT= Randomised control trial
What type of relevant info should we extract?
Study sample: Sample size, location, age, gender, ethnicity
Study design: Experimental → parallel, cross-over, factorial. Observational → cross-sectional, cohort, case-control
Potential for bias: Experimental → intention-to- treat, complete case analysis, blinding (has blinding occured? to what degree?). Observational → method of exposure measurement, with knowledge of outcome or without
Confounding: Large randomised control trials → allow balancing of confounders between groups, thus limiting the role of confounders
Measures of effect: Differ for qualitative and quantitative reviews. Direction and magnitude of effect. Dichotomous health outcomes. Relative risk, odds ratio, hazard ratio (SE, 95% CI)
Continuous health outcomes: Mean difference and associated variance (SE, 95% CI)
What is the ideal way to minimise the effects of confounding?
The ideal way to minimize the effects of confounding is to conduct a large RCT so that each subject has an equal chance of being assigned to any of the treatment options. If this is done with a sufficiently large number of subjects, other risk factors (i.e., confounding factors) should be equally distributed among the exposure groups. The beauty of this is that even unknown confounding factors will be equally distributed among the comparison groups :)
Observational studies prone to confounding
What else do we need to take into consideration during study analysis?
There are risk of bias tools to study the quality of the evidence; this can be high, moderate or low
Colour coding used with cochrane RoB 2 tool ⇒ red yellow green and each of the relevant literatures
Not only for meta analysis, but need to grade the quality of the evidence contributing to the meta analysis
What is the difference between clinicla and statistical heterogeneity?
There are 2 methods for stastical methods for pooling. These are the fixed effects and random effects model. Describe the fixed effects model
Fixed effects model
This model should be used only if it reasonable to assume that all studies shares the same, one common effect.
- Assume studies homogenous
- Greater relative weight to larger studies
- More conservative if small study / publication bias
- Similar to the Mantel-Haenszel method of pooling
There are 2 methods for stastical methods for pooling. These are the fixed effects and random effects model. Describe the random effects model
Random effects model assumes each study estimates a different underlying true effect, and these effects have a distribution (usually a normal distribution)
- Assume studies heterogenous
- Greater relative weight given to smaller studies
- Sources of heterogeneity need to be examined
- Contraindicated in the presence of publication bias
What is publication and small study bias?
- Publication bias= the failure to publish the results of a study on the basis of the direction or strength of the study findings - you are more likely to publish statistically significant results cf to non-significant conclusions, especially the case for smaller studies
- Some researchers may manipulate their results to ensure stastically significant results; may continue running statistical tests on a set of data until something statistically significant happens
Small study bias → smaller studies often get published to show sensational results but when you get larger studies (and probably more complete) show less sensational, and may even be null. roughly any study under 500 people would be considered a small study
What would this data look like on a forest plot?
https://canvas.sgul.ac.uk/courses/3661/pages/lecture-what-is-evidence-synthesis?module_item_id=198318
box size is related to the EFFECT SIZE
The metrics on the side determine the difference between the studies
NOTE - U NEED TO KNOW ABOUT HETEROGENETIY, SMALL STUDY BIAS AND USE OF RANDOM EFFECTS
