Critical Appraisal

Question 1

What is critical appraisal? What are the THREE factors that are assessed?

Answer 1

The process of carefully and systematically examining research to judge its trustworthiness, and its value and relevance in a particular context

Validity, results and clinical relevance are assessed

Question 2

What is PICO when talking about critical appraisal?

Answer 2

Population: Who are they? What are their characteristics?

Intervention: What was done? What drug was given at what dose?

Comparator: New intervention investigated, what was it compared to as stdnadard care?

Outcome: What was measured and what was found?

Question 3

What are the two questions that have to be asked before doing a critical appraisal?

Answer 3

1. Where is the trial published? –> High-quality peer-reviewed journals
2. Is the trial sponsored? –> Any conflicts of interest or bias that may affect how results are presented

Question 4

What are the four types of BIAS in clinical trials?

Answer 4

1. Selection bias
2. Performance bias
3. Detection bias
4. Attrition bias

Question 5

How to minimise bias?

Answer 5

• Effective randomisation
• Matching of participants
• Other treatments, measurements
• Blinding of participants and investigators

Question 6

Define the following abbreviations for the study methods of an critical appraisal;

A) R (population)

B) A

C) M

D) M,b,o (outcome)

Answer 6

A)

• Recruit a large enough sample PLUS Recruit subjects randomly
• OR Recruit consecutive patients

B)

• Allocate randomly to groups (and conceal allocation) OR Adjust for confounding

C)

• Manage groups equally
• Follow up all subjects

D)

• Measure outcomes with:
• blinded subjects and assessors and/or
• objective measures –> using numbers as much as possible

Question 7

What are the examples of descriptive studies used in critical appraisals?

Answer 7

• Mean ± SD for continuous variables (or median [range or IQR]) ƒ
• Number (%) for categorical variables

Question 8

What are the examples of comparative statistics used in critical appraisals?

Answer 8

• t-tests to compare means; ANOVA for 3 or more means
• x² analysis for categorical variables

Question 9

Explain the meaning of the following results

A) Relative risk (RR). RR = risk in the intervention group/ risk in the control group. What does a value of 1, <1 and > 1 mean?

B) Odds ratio (OR). OR = odds of an event occurring/ odds of it not occurring. What does a value of 1, <1 and > 1 mean?

C) Absolute risk reduction (ARR). ARR = risk of an event in the control group - risk in the intervention group. What does a value of 0, +ve and -ve mean?

D) Relative risk reduction.(RRR) RRR = 1 – ARR

E) Number needed to treat (NNT) NNT = 1/ARR

Answer 9

A)

• RR = 1. No difference
• RR < 1. Intervention reduces risk
• RR > 1. Intervention increases risk

B)

• OR = 1. No difference
• OR < 1. Exposure associated with reduced risk
• OR > 1. Exposure associated with increased risk

C)

• ARR = 0. No difference
• ARR +ve. Intervention beneficial
• ARR –ve. Intervention harmful

D)

• Reflects reduction in the rate of the event in intervention vs control group

E)

• Number of patients we need to treat to prevent 1 negative outcome; a small number is desirable (also consider NNH)

Question 10

What do checklists for critical appraisals assess?

Answer 10

• Safety
• Tolerability
• Efficacy
• Price