Meta Analysis Flashcards
Definition of meta-analysis
Technique for combining the findings from independent studies
Provides a precise estimate of the treatment effect, giving due weight to size of different studies included
Definition of heterogeneity
The presence of variation in true effect sizes underlying the different studies
Definition of clinical heterogeneity
Variation due to participant characteristics, types/timing of outcomes, measurements and intervention characteristics
Definition of statistical heterogeneity
Methodology differences between studies
Unknown study characteristics (method of randomization, different controls)
Definition of fixed effect model
Assumes 1 true effect size that underlies all studies
All differences in observed effects are due to sampling error
Definition of the random effects model
Assumes that the true effect varies from study tp study
Effect size is different due to varying participation of interventions
Definition of data synthesis
Combines estimates from selected studies using a fixed effect model or random effects model
Definition of publication bias
Bias related to what research is likely to be published among what is available to be published
Definition of tau
Compared to the p value, indication of consistency between estimates of effect
Definition of Z
The pooled effect that indicates the evidence or lack of for an effect
Where are meta analyses and RCTs in the hierarchy of evidence
Meta analyses are the highest level of evidence
RCTs are one level below
What is a meta analysis
Combines findings from independent studies and provides a precise estimate of treatment effect
Gives weight to the size of the different studies included
What 3 criteria must be met for a meta analysis to be valid
Quality of systematic view
Coverage of all relevant studies
Appropriate methods, considering heterogeneity, bias and sensitivity analysis of main findings
Before undertaking a systematic review, what should you do
Check if reviews
- exist
- are ongoing
Is a new one justified?
How would you use PICOS to structure your meta analysis
Population Intervention Comparator Outcome Study design
Used to generate a good research question
What is the protocol and what does it include
Document that describes the study conduct
Says what we want to do, must be followed
Includes
- Objectives (PICOS)
- Design
- Method
- Statistical considerations
- Methods of dissemination
What is heterogeneity and what are the 2 types
Presence of variation in true effect sizes underlying different studies
Can differ due to design, patients, interventions and outcomes
- Clinical heterogeneity
- Statistical heterogeneity
What is clinical and statistical heterogeneity
Clinical heterogeneity
-Differences in participant characteristics, types, timing out outcome, measurements and intervention characteristics
Statistical heterogeneity
- Methodological differences between studies
- Unknown study characteristics (method of randomization, different controls)
How would you assess and measure heterogeneity
Which stats test would you use
What is the degree of freedom
What is the p value
Null hypothesis, pooled effect is no different from study effect
x^2 squared used
df=no of studies-1
p-0,05, can be raised for small studies
What is the index of heterogeneity
% of total variations due to variation between studies
I^2 = 100 x [x^2 - df/x^2]
How do you interpret the I2 value and what can it tell you
0%, no heterogeneity
25%, low
50%, moderate
75%, high
Can be misleading as inconsistency depends on several factors
Effectiveness may vary, should studies be combined>
Pooled data may not reflect generalizable effectiveness
What are the 2 models that help you deal with heterogeneity
Fixed effect model
Random effect model
What is the fixed effect model
What does it assume
What are the problems when using this model
How do you calculate variance
How do you calculate weight
Assumes 1 true effect size that underlies all studies
All differences in observed effects due to sampling error
Open to bias, has narrow CI and may not be appropriate if heterogeneity present
Variance = SE^2 Weight = 1/variance
What is the random effects model
What does it assume
What kind of studies are included
How do you calculate variance
Assumes that the true effect varies from study to study
Effect size is different due to varying participants, age, intervention
Studies included in analysis assumed to be a random sample of all possible studies that meet inclusion criteria
Variance = SE^2 + intertrial vranke (tau^2)
What is publication bias
What is more likely to be published
Why is publication bias a problem
How would you assess publication bias
Bias related to what research is likely to be published among what is available to be published
Research with significant findings regardless of quality
By combing only published research => overestimate of effect size
Funnel plots
Effect size against sample size/other indicators of estimate prediction (SE)
How would you interpret a funnel plot
95% of studies expected to lie within the limit lines
Smaller SE => higher precision
If symmetrical
-Estimates are around the average
If asymmetrical
- Can show publication bias
- Or small study effect
Describe how the sample size can affect the outcome
When are symmetry tests useful
Interventions tested in small studies may differ from those tested in large studies
Symmetry tests exist but aren’t effective when study no is small
What are the 5 search strategies for research
Cochrane Register Conference proceedings Trial and research registers Contacted trialists, experts and researchers Manufacturers of commercial devices
Allows you to check both published and unpublished work
What are the 2 types of outcome measure
What is the difference between both of them
Primary outcomes
- outcome that is the most important out of the many outcomes
- must be defined at the start of the study
- must also state how the outcome would be measured
Secondary outcomes
- Planned outcome measure that is not as important but is of interest
- good to define exactly what be considered a secondary outcome
How is the data collected and analysed
2 review authors independently select trials for inclusion, assessed trial quality bias risk and extract data
If either one is involved in a study, a 3rd review author handles the data
Trialists contacted for more info
Results analyses as standardized mean difference (SMD) if continuous/risk differences (RD) for dichotomous variables
What data is extracted and how is it managed
Checklist used to note
- methods of generating randomization
- methods of concealment of allocation
- blinding of assessors
- ITT
- AE and dropout reasons
- participant properties
- duration of treatment
- comparison between intervention and controls
Describe how you would use a forest plot in data display What does -Tau -I2 -Z mean
Studies represented by author and pub date
Mean, SD and total no of participants in both control and experimental data
Study results visually and numerically presented
- bigger square = more meaningful studies with large sample size and smaller CI
- wider CI => decreased reliability
- diamond, estimate of pooled effect
Weight of each data set
Tau = tests for consistency between estimates of effect I2 = proportion of variation in study estimates due to heterogeneity Z = test for pooled effect
What can happen in a subanalysis
All data can be split into subgroups according to patient properties and meta analysis performed on subsets
What would you include in the conclusion
Based on results of pooled data
Results must be interpreted with caution due to heterogeneity
-varying quality
-variation in intensity, duration and treatment
-variation in patient properties
What is the purpose of a forest plot
Shows heterogeneity/inconsistency between studies
How else would you detect heterogeneity sources
Sensitivity/subgroup analysis
What is the purpose of a funnel plot
Detect effect of study size and possible publication bias
What sort of studies get more weight
Emphasis on RCTs due to fewer biases
