Meta Analysis Flashcards by An Nakamura

Definition of meta-analysis

Technique for combining the findings from independent studies
Provides a precise estimate of the treatment effect, giving due weight to size of different studies included

How well did you know this?

Not at all

Perfectly

Definition of heterogeneity

The presence of variation in true effect sizes underlying the different studies

How well did you know this?

Not at all

Perfectly

Definition of clinical heterogeneity

Variation due to participant characteristics, types/timing of outcomes, measurements and intervention characteristics

How well did you know this?

Not at all

Perfectly

Definition of statistical heterogeneity

Methodology differences between studies

Unknown study characteristics (method of randomization, different controls)

How well did you know this?

Not at all

Perfectly

Definition of fixed effect model

Assumes 1 true effect size that underlies all studies

All differences in observed effects are due to sampling error

How well did you know this?

Not at all

Perfectly

Definition of the random effects model

Assumes that the true effect varies from study tp study

Effect size is different due to varying participation of interventions

How well did you know this?

Not at all

Perfectly

Definition of data synthesis

Combines estimates from selected studies using a fixed effect model or random effects model

How well did you know this?

Not at all

Perfectly

Definition of publication bias

Bias related to what research is likely to be published among what is available to be published

How well did you know this?

Not at all

Perfectly

Definition of tau

Compared to the p value, indication of consistency between estimates of effect

How well did you know this?

Not at all

Perfectly

Definition of Z

The pooled effect that indicates the evidence or lack of for an effect

How well did you know this?

Not at all

Perfectly

Where are meta analyses and RCTs in the hierarchy of evidence

Meta analyses are the highest level of evidence

RCTs are one level below

How well did you know this?

Not at all

Perfectly

What is a meta analysis

Combines findings from independent studies and provides a precise estimate of treatment effect
Gives weight to the size of the different studies included

How well did you know this?

Not at all

Perfectly

What 3 criteria must be met for a meta analysis to be valid

Quality of systematic view
Coverage of all relevant studies
Appropriate methods, considering heterogeneity, bias and sensitivity analysis of main findings

How well did you know this?

Not at all

Perfectly

Before undertaking a systematic review, what should you do

Check if reviews

exist
are ongoing

Is a new one justified?

How well did you know this?

Not at all

Perfectly

How would you use PICOS to structure your meta analysis

Population
Intervention
Comparator
Outcome
Study design

Used to generate a good research question

How well did you know this?

Not at all

Perfectly

What is the protocol and what does it include

Document that describes the study conduct
Says what we want to do, must be followed

Includes

Objectives (PICOS)
Design
Method
Statistical considerations
Methods of dissemination

What is heterogeneity and what are the 2 types

Presence of variation in true effect sizes underlying different studies
Can differ due to design, patients, interventions and outcomes

Clinical heterogeneity
Statistical heterogeneity

What is clinical and statistical heterogeneity

Clinical heterogeneity
-Differences in participant characteristics, types, timing out outcome, measurements and intervention characteristics

Statistical heterogeneity

Methodological differences between studies
Unknown study characteristics (method of randomization, different controls)

How would you assess and measure heterogeneity
Which stats test would you use
What is the degree of freedom
What is the p value

Null hypothesis, pooled effect is no different from study effect
x^2 squared used
df=no of studies-1
p-0,05, can be raised for small studies

What is the index of heterogeneity

% of total variations due to variation between studies

I^2 = 100 x [x^2 - df/x^2]

How do you interpret the I2 value and what can it tell you

0%, no heterogeneity
25%, low
50%, moderate
75%, high

Can be misleading as inconsistency depends on several factors
Effectiveness may vary, should studies be combined>
Pooled data may not reflect generalizable effectiveness

What are the 2 models that help you deal with heterogeneity

Fixed effect model

Random effect model

What is the fixed effect model
What does it assume
What are the problems when using this model

How do you calculate variance
How do you calculate weight

Assumes 1 true effect size that underlies all studies
All differences in observed effects due to sampling error

Open to bias, has narrow CI and may not be appropriate if heterogeneity present

Variance = SE^2
Weight = 1/variance

What is the random effects model
What does it assume
What kind of studies are included

How do you calculate variance

Assumes that the true effect varies from study to study
Effect size is different due to varying participants, age, intervention

Studies included in analysis assumed to be a random sample of all possible studies that meet inclusion criteria

Variance = SE^2 + intertrial vranke (tau^2)

What is publication bias What is more likely to be published Why is publication bias a problem How would you assess publication bias

Bias related to what research is likely to be published among what is available to be published Research with significant findings regardless of quality By combing only published research => overestimate of effect size Funnel plots Effect size against sample size/other indicators of estimate prediction (SE)

How would you interpret a funnel plot

95% of studies expected to lie within the limit lines Smaller SE => higher precision If symmetrical -Estimates are around the average If asymmetrical - Can show publication bias - Or small study effect

Describe how the sample size can affect the outcome | When are symmetry tests useful

Interventions tested in small studies may differ from those tested in large studies Symmetry tests exist but aren't effective when study no is small

What are the 5 search strategies for research

``` Cochrane Register Conference proceedings Trial and research registers Contacted trialists, experts and researchers Manufacturers of commercial devices ``` Allows you to check both published and unpublished work

What are the 2 types of outcome measure | What is the difference between both of them

Primary outcomes - outcome that is the most important out of the many outcomes - must be defined at the start of the study - must also state how the outcome would be measured Secondary outcomes - Planned outcome measure that is not as important but is of interest - good to define exactly what be considered a secondary outcome

How is the data collected and analysed

2 review authors independently select trials for inclusion, assessed trial quality bias risk and extract data If either one is involved in a study, a 3rd review author handles the data Trialists contacted for more info Results analyses as standardized mean difference (SMD) if continuous/risk differences (RD) for dichotomous variables

What data is extracted and how is it managed

Checklist used to note - methods of generating randomization - methods of concealment of allocation - blinding of assessors - ITT - AE and dropout reasons - participant properties - duration of treatment - comparison between intervention and controls

``` Describe how you would use a forest plot in data display What does -Tau -I2 -Z mean ```

Studies represented by author and pub date Mean, SD and total no of participants in both control and experimental data Study results visually and numerically presented - bigger square = more meaningful studies with large sample size and smaller CI - wider CI => decreased reliability - diamond, estimate of pooled effect Weight of each data set ``` Tau = tests for consistency between estimates of effect I2 = proportion of variation in study estimates due to heterogeneity Z = test for pooled effect ```

What can happen in a subanalysis

All data can be split into subgroups according to patient properties and meta analysis performed on subsets

What would you include in the conclusion

Based on results of pooled data Results must be interpreted with caution due to heterogeneity -varying quality -variation in intensity, duration and treatment -variation in patient properties

What is the purpose of a forest plot

Shows heterogeneity/inconsistency between studies

How else would you detect heterogeneity sources

Sensitivity/subgroup analysis

What is the purpose of a funnel plot

Detect effect of study size and possible publication bias

What sort of studies get more weight

Emphasis on RCTs due to fewer biases