Critical Appraisal

Question 1

Levels Of Evidence

Answer 1

1a - Systematic review of RCTs
1b - Individual RCTs with narrow confidence intervals
2a - Systematic reviews of cohort studies
2b - Individual cohort studies and low-quality RCTs
3a - Systematic review of case-control studies
3b - Case-control studies
4 - Case series and poor-quality cohort and case-control studies
5 - Expert opinion

Question 2

Information about Study method

Answer 2

• Who was recruited? (Age/sex/ethnicity)
• What was the inclusion and exclusion criteria?
• How were participants assigned to the two intervention arms? (random/blinded)
• What was the intervention?
• How was the data interpreted. What statistical analysis was used?

Question 3

Interpretation of results

Answer 3

3 possible explanations for results reported in a study:
- True results
- Bias and confounding
- Chance

Experts agree that a relative risk must be > 3 to be clinically significant.

Question 4

Definition of results terminology

Answer 4

Odds Ratio OR- #events relative to #non-events in each group. OR > 1 means odds have increased. OR < 1 is not significant.

Relative Risk RR - Ratio of risks of the treated group relative to control group
Relative Risk Reduction RRR = 1- RR

Absolute risk reduction ARR - Difference between risk of an event in control group and risk of event in treated group.

Number needed to treat NNT - Reciprocal of ARR - Number of patients that need to be treated to get the desired outcome in one patient who would not have benefited otherwise.

P-value - Possibility that results would have occurred by chance. (e.g. 0.05 = 5% = 1 in 20). P <= 0.05 is statistically significant.

Confidence interval = Range of values around the estimated effect has a high probability of containing the true value. The larger the sample size, the smaller the confidence interval.

Intention to treat analysis - Participants are analysed in the groups to which they were randomised even if they discontinue the trial or cross over to another group. Combats attrition bias.

Question 5

Conclusions considerations

Answer 5

• Is this study valid?
  
  • Internal validity - Representative of true association between exposure and outcome. Depends on study design, implementation, data analysis, elimination of bias.
  • External validity - Are findings able to be generalised to other groups or populations - Reflected in inclusion and exclusion criteria.
• Is this study reliable? - Will results be reproducible in similar studies
• Are the results clinically relevant?

Question 6

Types of bias

Answer 6

• Selection - Subject allocation results in treatment groups that are systematically different.
• Detection - Measurements are taken differently between treatment groups
• Observer - Data collector is able to be subjective about the outcome.
• Publication - Negative studies are more likely to be submitted than positive ones.
• Recall - Altered reporting of symptoms by patients relating to different allocation of groups.
• Response - Patients who enrol in trial differ from the population.
• Hawthorne effect - Process of doing the study improves the outcome.