Week 10- Introduction to Evidence

Question 1

What is critical appraisal?

Answer 1

Process of systematically examining research evidence to assess its validity, results and relevance before using it to inform a decision

Question 2

What does critical appraisal do?

Answer 2

• Evaluate validity of research
• Methodology of research: internal validity
• Generalisability of research: external validity
• Weigh up evidence to see how useful it is in clinical decision making
• Balanced assessment of benefits and strengths of research evidence

Question 3

Why should a clinician undertake critical appraisal?

Answer 3

• Helps you identify methodological flaws and assess impact on findings
• Identify ways of improving research methodology and providing more believable results

Question 4

What are quality dimensions of research evidence?

Answer 4

1) Hierarchy level
- Study design: strength of evidence based on design of research
2) Study quality
- How good is the study: strength of evidence based on methodology
3) Statistical precision of results
- Statistical significance (p value, confidence intervals): the strength of evidence based on how much chance played a role
4) Effect size
- How clinically important and the findings (impact)
5) Relevance
- Usefulness of results in clinical practice

Question 5

What are the two approaches to appraise research?

Answer 5

• Assessment of risk of bias: can results be believed whilst weighing up bias
• Assessment of methodological quality: extent to which researchers conducted research to highest standards

Question 6

How do you appraise primary research?

Answer 6

Research methodology
-Research question focused and original
-Was design/method appropriate 
-How was research undertaken
-How were date collected and analysed
Generalisability of research
-Relevance of patient population

Question 7

How do you appraise RCTs?

Answer 7

• Recruitment and sample size (sampling)

- Randomised method and controls (confounding factors, blinding, follow-up)

Question 8

How do you appraise cohort/case control?

Answer 8

• Recruitment: selection bias
• Exposure: measurement, recall or classification bias
• Confounding factors & adjustment
• Time frames

Question 9

How do you appraise qualitative research?

Answer 9

• Sampling
• Triangulation of methods
• Independent analysis
• Reflexivity
• Respondent validation
• Plausible interpretation
• Saturation

Question 10

How do you appraise systematic reviews?

Answer 10

• Clear review question (PICO, PECOT, PICo)
• Thorough literature search carried out
• Quality of studies assess
• If results were combined, was this appropriate (meta-analysis)

Question 11

How do you use critical appraisal tools? (CAT)

Answer 11

-Choice of CAT depends on research question, reason for undertaking critical appraisal, types of research identified

Question 12

What are common misconceptions of critical appraisal?

Answer 12

• Negative dismissal of research
• Assessment purely based on results alone
• Based entirely on statistical analyses utilised
• Undertaken by researchers or experts in field

Question 13

What are two widely accepted CATs?

Answer 13

Domain based
-Cochrane to evaluate risk of bias
Scales or checklist
-Comparison across multiple studies
-Provides a numerical score, indicates methodological quality of research