Data Analysis and Interpretation

Question 1

AMSTAR2 Steps (Data Analysis and Interpretation)

Answer 1

8. Were studies described
9. Was risk of bias included
10. Was source of funding reported
11. Were results properly combined
12. Was impact of risk of bias assessed
13. Was risk of bias considered when interpreting results
14. Was heterogeneity explained
15. Was publication bias investigated
16. Any source of conflict of interest

Question 2

Why should studies be described

Answer 2

Allows us to better understand the population, intervention, control, and outcomes
- Also the study design, analyses , and setting

Lets us determine if the study is a good fit

Question 3

When to consider bias

Answer 3

When selecting studies
- Selection performed in duplicate
- Justifying excluding studies
- Investigating publication bias

When assessing quality of included studies

Question 4

Satisfactory technique

Answer 4

Use the correct tools
- Assessment tool for randomized trials should be used to review randomized trials
- Assessment tool for observational studies should be used to review cohort studies

Question 5

How to show risk of bias on included studies

Answer 5

Minimum: Report bias assessment for each study

Common: Quality score in study characteristics table

Ideal: Supplementary table with detailed review of each risk of bias item

Question 6

Methods of combining data

Answer 6

Pooled odds ratio
- Outcome measured as dichotomous variable

Weighted mean difference
- Outcome is measured as a continuous variable

Question 7

Methods for combining data to calculate a calculating a pooled odds ratio

Answer 7

Fixed Effects Model (Mantel-Haenszel)

Random Effects Model (DerSimonian & Laird)

Question 8

Fixed Effects Model

Answer 8

Assumes treatment effect is the same across all studies
- Variance all comes from within studies
- Any variance between studies is due to randomness

Larger studies have a greater influence
- Creates narrow confidence intervals

Question 9

Random Effects Model

Answer 9

Assumes that included studies are a random sample from a population of studies addressing the posed question
- Variance comes from both within and between studies

Larger studies still have a greater influence, but, the gradient of influence between large and small studies is smaller
- Creates wider confidence intervals

Question 10

Homogenicity

Answer 10

Consistient, uniform characteristics across all studies

Question 11

Heterogenicity

Answer 11

Substantial differences in characteristics across all studies

Question 12

Key question to ask if there are observed differences between studies

Answer 12

Is it being caused by random chance (Random Error)
- Solve by increasing sample size

Is it being caused by an underlying problem (Systematic Error/Bias)

Question 13

How to identify between-study differences

Answer 13

Visually

Statistical Analysis
- P-value
- I^2 Test

Question 14

P-Value

Answer 14

If less than 0.05
- Heterogenous: Between study differences are significant

If greater than 0.05
- Homogenous: Between study difference are not significant

Question 15

I^2 Test

Answer 15

Lower values indicate less variation across studies
- I^2 less than 50 is considered okay

Anything higher would be considered heterogenous

Question 16

Meta Analysis vs Network Meta Analysis

Answer 16

Meta Analysis
- Good for direct comparisons between one intervention and a reference group
- Which is better?

Network Meta Analysis
- Good for direct and indirect comparisons of multiple interventions and comparisons
- Which is best?