Session 6: Reviews of Evidence

Question 1

Features of narrative reviews.

Answer 1

Implicit assumptions

Opaque methodology

Not reproducible leading to bias and subjectivity

Question 2

Give features of systematic reviews.

Answer 2

A clearly focused question

An explicity statement about type of study, type of participants, types of interventions, types of outcome measures.

A systematic literature search

A selection of materials

Appraisal

A synthesis possible including a meta-analysis.

Question 3

Key features of systematic reviews.

Answer 3

Explicit assumptions

Transparent methodology

Reproducible leading to unbias and objectivity

Question 4

What is a meta-analysis?

Answer 4

A quantitative synthesis of the results of two or more primary studies that addressed the same hypothesis in the same way.

Question 5

Difference between systematic review and meta-analysis.

Answer 5

A systematic review will not necessarily include a meta-analysis but more or less all MA have systematic reviews.

Question 6

Give an example of when systematic review won’t include MA.

Answer 6

When clinical heterogeneity is too great

Question 7

Purpose of meta-analysis.

Answer 7

Facilitate synthesis of a large number of study results.

Systematically collate study results

Reduce problems of interpretation due to variations in sampling

Question 8

Quality criteria of MA.

Answer 8

Compilation of complete set of studies

Identification of common variable or category definition.

Standardised data extraction

Analysis allowing for sources of variation

Question 9

Calculate the odds ratio of surviving.

Answer 9

Aspirin: 566/49 = 11.55:1

Placebo: 557/67 = 8.31:1

Odds ratio = 11.55/8.31 = 1.39

Question 10

Interpret the result of the odds ratio being 1.39.

Answer 10

1.39 more likely to survive if you are on aspirin then on placebo.

Question 11

The odds ratio is 1.39 and the error factor is 1.48.

Calculate the confidence interval.

Answer 11

1. 39 x 1.48 = 2.05
2. 39/1.48 = 0.94

The confidence interval is 0.94 - 2.05

Question 12

The confidence interval is 0.94 - 2.05

Is the result statistically significant?

Answer 12

No.

The null hypothesis (1) is within the confidence interval which means that the result could be due to chance.

Question 13

Meta-analysis calculated a pooled estimate of odds ratio.

Explain what this means.

Answer 13

Odds ratio and their 95% CIs are calculated for each study.

They are then combined to give a pooled estimate odds ratio using a statistical computer program.

Question 14

In the pooled estimate the individual studies are weighted.

Weighted according to what?

Answer 14

Their size

The uncertainty of their odds ratio

Question 15

What does a large size of a study suggest in weight?

Answer 15

Greater weight

Question 16

What does a narrow CI suggest according to weight?

Answer 16

A narrow CI = greater weight

Question 17

How are meta-analysis visualised?

Answer 17

By a forest plot.

Question 18

What does the size of the square indicate?

Answer 18

Weight of the study

Question 19

What does the line going through each square indicate?

Answer 19

The CI of that study

Question 20

What does the vertical line indicate?

Answer 20

The null hypothesis

Question 21

What does the width of the diamond indicate?

Answer 21

The confidence interval of the pooled estimate

Question 22

What does the vertical dotted line indicate?

Answer 22

The pooled odds ratio

Question 23

Problems with meta-analysis

Answer 23

Heterogeneity between studies

Variable quality of the studies

Publication bias in selection of the studies

Question 24

What are the two models in order to take heterogeneity into account?

Answer 24

Fixed effect model

Random effect model

Question 25

Explain fixed effect model

Answer 25

Assumes that the studies are estimating **exactly** the same **true effect** size. There is only one true effect in fixed effect model

Question 26

Explain random effect model.

Answer 26

Assumes that the studies are **estimating similar,** but not the same true effect size. There is **mean** true effect

Question 27

Pooled estimate OR in fixed effect model was 1.11. Pooled estimate OR in random effects model was 1.14. The between study variance (t2) was 0.01. What does this mean?

Answer 27

That the heterogeneity between the studies was low.

Question 28

How does the odds ratio differ in fixed vs random effect?

Answer 28

It is often very similar.

Question 29

How does the 95% CI differ in fixed vs random effect?

Answer 29

Often wide in random effect vs fixed effect.

Question 30

How does the weighting of the studies differ in random effects vs fixed effect model?

Answer 30

More equal between studies in the random effects model which means that there is greater weighting towards small studies.

Question 31

A test for heterogeneity has a p value of 0.082. The p-value is regarded significant if the p-value is less than 0.1. What does this mean?

Answer 31

This means that we can reject the null-hypothesis which is that there is heterogeneity. In other words this means that there is **homogeneity.**

Question 32

Limitations of random effects model and heterogeneity.

Answer 32

Random effects model can only account for variation but **not** explain it.

Question 33

What is used to explain heterogeneity?

Answer 33

Sub-group analysis

Question 34

Give examples of variable quality.

Answer 34

Poor study design Poor design protocol Poor protocol implementation

Question 35

Rank studies in how prone they are to bias. (More prone higher up)

Answer 35

Case-control Cohort Non-randomised controlled trials Randomised controlled trials

Question 36

Approaches to analyse variable quality in studies.

Answer 36

# Define a basic quality standard and only include studies that satisfy the criteria. Score each study for its quality and then incorporate the quality score into weighting, or use sub-group analyses to explore differences (high quality studies vs. low quality studies)

Question 37

Give examples of what to assess in RCTs.

Answer 37

Allocation methods (was it randomised?) Blinding and outcome assessment Intention-To-Treat Appropriate statistical analysis

Question 38

What is publication bias?

Answer 38

Studies with **statistically** **significant** or favourable restults are more likely to be published than those studies with non-statistically significant or unfavourable results. This applies particularly to smaller studies.

Question 39

What does publication bias mean in terms of systematic review or meta-analysis?

Answer 39

That they can be flawed by such bias. Publication bias leads to a biased selection of studies towards demonstration of effect.

Question 40

What is used in order to identify whether there is publication bias or not?

Answer 40

Plot results of identified studies against a measure of their size via a **funnel plot**.