L13. Systematic Reviews and Meta-analyses

Question 1

What is the Cochrane Collaboration and why does it exists?

Answer 1

A group formed to organise medical research information systemically to aid in evidence based medicine. They conduct systematic reviews of randomised controlled trials.

Question 2

What is a Systematic Review? And what are the four main aims of the systematic review?

Answer 2

It is a literature review that focuses on answering a single question.
It aims to identify, appraise, select and synthesise high quality evidence to answer a question.

Question 3

What is important to a systematic review?

Answer 3

A well-defined criteria for the review

Question 4

What kind of data do systematic reviews analyse?

Answer 4

Both Clinical Trial (mainly) and Observational data

Question 5

What kind of analysis gives the highest level of incidence? Is this always the case?

Answer 5

Systematic reviews often give the highest level of evidence of the study types.
However, a well conducted randomised controlled trail that is very rigorous may have the potential to have a higher level of evidence.

Question 6

What is a meta-analysis?

Answer 6

The statistical aspect of a systematic review: analysis of the combined data from multiple studies

Question 7

How is the meta-analysis data pooled?

Answer 7

Derived pooled into a weighted average based on the size of the effect of the findings.

Question 8

What are the 4 purposes of a meta-analysis

Answer 8

1. Increase the power
2. Resolve uncertainty
3. Improve estimates of effect size (increase precision)
4. Answer other questions

Question 9

How do we identify what studies to use?

Answer 9

Having a clear clinical or research question that you want to answer based on studies.
PICOT analysis

Question 10

What are some data sources to use to find studies? [3]

Answer 10

Medline, embase, central and other databases
Reference lists of relevant papers
Others like ‘grey literature’ or websites

Question 11

What important considerations must be made when searching through databases?

Answer 11

Using standard (MESH-medical subject headings) terms where possible, include all relevant spellings and types of terms, limits (eg. only including randomly controlled trials in the search).

Question 12

What is ‘grey literature’ and why may some of the findings from these be important?

Answer 12

Studies conducted that are unpublished. These are important as they may be just as rigorous in nature as the published studies and may hold relevant information like negative results or null results which are just as important in a systemic review.

Question 13

How do we decide on what to include and exclude from the systematic review? (what criteria)

Answer 13

Whether they are in line with the question being asked
PICOT parameters
Other parameters (like sample size)

Question 14

It is important to not introduce bias when selecting for papers to include/exclude from systemic analyses. Give 2 examples where bias may be introduced

Answer 14

1. When screening for larger sample sizes in studies (these have higher power) you may be introducing selection bias as larger studies often com from larger centres with funding and aiming for a certain result. You may be discounting important information from smaller funded research.
2. Studies published in different languages are sometimes excluded for purposes of convenience but they may contain very important and relevant information

Question 15

What are the steps [5] in selecting and appraising studies?

Answer 15

1. At least 2 people independently start the search
2. Reading all abstracts that are relevant (important to note that abstracts don’t always reflect the quality of the study).
3. Apply the inclusion and exclusion criteria to abstracts
4. Obtain full papers
5. Asses the papers (particularly the methods) for quality

Question 16

What are Consolidated Standards of Reporting Trails? What can the include?

Answer 16

Guidelines are available to help analyse the quality of a paper?
They often include checklists and flow charts

Question 17

What is an important thing to asses in studies when conducting systematic reviews? What is the inherent risk in this.

Answer 17

The risk of bias and confounding: this in itself is a subjective measure

Question 18

How do we undertake meta-analysis?

Answer 18

Statistical software and analysing how statistical analysis was undertaken in different studies.

Question 19

What is the concept of a weighted average?

Answer 19

Weighted average, also called EFFECT SIZE results from the multiplication of each component by a factor reflecting its importance.

Question 20

How do we achieve a weighted average in meta-analysis?

Answer 20

Studies are assessed and compared to according to the quality of data each study had. And then from this extrapolate how much of an effect or weight (% or proportion wise) that study will affect the meta-analysis. The higher the quality, the larger its weighting

Question 21

What determines the weighting of individual studies?

Answer 21

Sample size and Inverse variance

Question 22

What is inverse variance?

Answer 22

Variance is the deviation of the outcome of the study from the mean outcome of the combined study analysis. The greater the variance, the lower the weighting it will have in the overall meta analysis. Thus the weighting is inversely proportional to the variance.

Question 23

What is meant by heterogeneity of the studies in meta-analysis? What is desired?

Answer 23

Heterogeneity is the variability in the effect sizes (weighted average) in terms of whether they similar to one another in nature to be pooled. The HIGHER the P value the less likely they are different so this is a good number (high similarity).

Question 24

What is a forest plot? What kind of information does it include?

Answer 24

A quick way to record the information from a systematic review.
It includes a test for heterogeneity, the line of the null effect of each study and their confidence intervals (size of the square is the weighing of the study and the width indicates confidence interval), the pooled effect (shown by diamonds).

Question 25

What are some considerations in deciding heterogeneity?

Answer 25

Statistical analysis: comparing effect sizes and variances

Non-statistical: PICOT (cannot be objectively assessed)

Question 26

How do we report systematic reviews and meta analysis?

Answer 26

In a number of standardised ways outlined in PRISMA guidelines (Preferred Reporting Items for Systemic Reviews and Meta-analysis) which gives flow charts and checklists.