CAT Systematic Review

Question 1

SR
Why is this study type appropriate to match the clinical question? (3)

Answer 1

• The SRs includes multiple (insert type of study here) which are most appropriate to answer the the clinical question.
• E.g. Treatment efficacy/intervention = RCT. Aetiology = cohort Prognosis/outcomes = case-control.*
• The benefit of an SR is to bring together all the available evidence
• Combined with a meta-analysis it gives greater precision on estimating effectivness, than from individual studies alone.

Question 2

How did the researchers identify which papers form part of the study? (3)

Answer 2

• State inclusion criteria.
• Pick out good things and parts they missed
• State the importance of a good search strategy.

A good inclusion strategy should involve:
1. Inclusion of grey literature/unpublished studies
2. No language restrictions
3. Studies that align with the clinical question (aka have 1+ outcome of interest).
4. Only studies specified by the SR to include (e.g only RCT)
5. Personal contact with experts
6. Multiple search databases
7. Hand-searching / snowballing for references

In order to minimise selection bias–> to secure the external validity of the results.

Question 3

How was the quality of the papers assessed? S&Ws (3)

Answer 3

Quality assessment should include:
1. Standardised checklists (Cochrane)
2. Mutiple reviewers in screening process
3. A third party for any disagreements
4. Scoring system for study quality.

The result of this is …
a replicable process of quality assessment.
Quality assessmnet prevents differential errors which could reduce both internal/external validity of results.

Question 4

Within the quality assessment process, explain the rationale for assessing the following:
* Concealment allocation
* Blinding of investigators
* Completeness to follow-up
* Intenstion to treat analysis

Answer 4

• Concealment allocation - prevents selection bias during recruitment & randomisation stage.
• Blinding of investigators/participants - prevents observor or expectation bias after randomisation stage (during the trial itself).
• Completeness to follow-up - lossess to follow up –> reduced power. Bias if more losses in one group vs another.
• Intention to treat analysis - must be carried out even if they are lost to follow-up –> gives a pragmatic estimate of the treatment effect.

Question 5

Define:
Statistical heterogeneity
Clinical heterogeneity

Answer 5

• How the estimate of efficacy of a (treatment) differs between the individual studies
• How the types of treatment vary between individual studies.

Question 6

What is a meta-analysis? (2)
Aka what is it and whats the benefit?

Answer 6

1. A meta-analysis uses pooled results from multiple studies to generate an average estimate (of the treatment efficacy).
2. Increases precision of the estimate.

Question 7

For some outcomes meta-analysis may not be able to be used. Why? (2)

Answer 7

Meta-analysis can only be carried out if:
1. There are 2+ studies which include the same outcome.
2. There is low statistical heterogeneity (aka data is homogenous so can be collated)

Both ensure the reliability of results.
If data is combined that is too heterogenous the analysis could be meaningless/misleading.

Question 8

How are the results presented and what is the main result?
How precise are the results?

Answer 8

• What is the bottom line result
• What is the result?
• How were they expressed? (NTT, OR within forest plot)

Precision
* Confidence intervals: narrow (increases precision) or wide(reduces precision)?
* Are results statistically significant (CI doesn’t include the null relationship = yes)

Question 9

Forest plots - usual way of displaying data from a meta-analysis.
What is the meaning of the:
- Vertical Line
- Individual squares
- Individual horizontal lines
- Diamond
- Heterogeneity score & associated p value
- Weighting

Answer 9

• “Line of no effect”: aka RR=1. No difference between treatment vs control for the outcome.
• Individual results for each study; size correlates with precision = larger sample size.
• Confidence intervals (not statistically significant if cross the line of no effect)
• The average estimate combining all the studies: width = 95% CI
• Heterogenity score, I2: low percentage, large p value >0.1 = studies likely to be measuring the same thing
• the influence of an indivudal study on the overall result (closely related to sample size).

Question 10

95% confidence intervals.

Answer 10

• Credible margin of error (for each study estimate)
• we can be 95% certain that the true value lies within (lower limit) and (upper limit).

Question 11

Can the resutls be applied to local population? - AKA External Validity

Answer 11

• Patients in SR sufficiently similar to patient in scenario?
  Consider outcomes from the point of view of the:
• Individual - Age, Sex, Outcome/comparator (disease in question)
• Policy makers and professionals - e.g. NHS? Costs?
• Patients family/carers
• Local setting - different/similar? (hospital, community, country differences)?
• Wider community

Should policy/practice change as a result of this trial?
Have all important harms/benefits been cinluded?

Question 12

Were all important outcomes considered ?

Answer 12

• What other info would you want to be included?
• Were there subgroup analyses?