Critical appraisal tutorial

Question 1

What should you consider when appraising the question

Answer 1

what is the question the researchers are trying to answer? Is the clinical issue being addressed important? Is the research original in terms of its size, methods, study population etc?

Question 2

What should you consider when appraising the design

Answer 2

What design was used? Refer back to the hierarchy of evidence, and consider whether the correct design was used to answer the question asked.

Question 3

What should you consider when appraising the population

Answer 3

sample size/power calculations, recruitment of participants, inclusion/exclusion criteria, are the results generalisable? Where the subjects studied in real life situations?

Question 4

What should you consider when appraising the methods

Answer 4

what specific intervention was being considered and what was it being compared with? What outcome was measured and how? Has the study continued for long enough to detect the effect of the intervention

Question 5

What should you consider when appraising the analysis

Answer 5

did the authors undertake appropriate statistical analysis? Assess chance? Adjust for confounding?

Question 6

What should you consider when appraising confounding

Answer 6

re there confounders which have not been adjusted for? (LOTS of things can be considered here e.g., demographics of usual supplement consumers, lifestyle choices etc

Question 7

What should you consider when appraising bias

Answer 7

Do the authors interpret their findings correctly? Could there be any biases they have missed? Do they make causal inference (Bradford-Hill)?

Question 8

What should you consider when appraising ethics

Answer 8

Is the study question ethical? Was informed consent obtained from all participants? Was the trial approved by a Research Ethics Committee? Was the trial registered and regulated by an independent scientific committee?

Question 9

What should you consider when appraising interpretation

Answer 9

• There are huge methodological questions with regard nutritional research as a whole – E.g., Do supplement users constitute a more financially secure, health conscious, generally ‘healthy living’ population? Can we be certain that it is the intake from the supplements (and not something else) which is causing the effects observed (if any)? Does agreement to participate reflect an unusual or biased population sample? What evidence do we have that the supplements studied are the cause of the effects observed?
• Is 10 years of follow up enough when considering mortality/morbidity? And is recruitment done early enough? If we develop fatty streaks in our arteries before 2 years of age, why only study people >40 years?
• What are the effects of care on mortality analyses? This information is very difficult to quantify and subjective to collect
  Etc!

Question 10

What is the difference between CONSORT and STROBE

Answer 10

a. CONSORT Statement 2001 and eFlowchart 2005: Items to include when reporting a randomized trials;
b. STROBE Statement: Checklist of items that should be included in reports of observational studies;

Question 11

Ultimately, why are critical appraisal skills important in medicine

Answer 11

They are a core part of clinical practice – finding, making sense of, and applying new research evidence to enhance practice
They help to improve communication with patients – providing informed advice to patients asking about new treatments they have seen on the internet increasing survival or not etc