14 Introduction to Critical Appraisal and How to Read Published Papers

Question 1

Q: It is important to consider first why you are doing a critical review. Typically, they’re carried out with the aim of… (2)

Answer 1

A: • Complete a deep review of a single paper, questioning whether the work reported has been planned, conducted, and presented in an appropriate manner; or
• To inform your learning about a question you have to
answer. This is not usually the same as that being
investigated (and reported) by the authors of the paper
being reviewed. Rather, you are using the evidence they
have presented to determine whether the rationale for
your question (or address thereof) is appropriate, and fully inclusive of learning reported by others.

Question 2

Q: How do you systematically find references?

Answer 2

A: Start by writing a clear question which you aim to answer using the papers found in your literature search.

Once this is done, consider what ‘key words’ might be used by authors publishing in this area, and use these to search for appropriate papers.

Question 3

Q: How do you deal with finding more papers than you can cope with?

Answer 3

A: refine your search (e.g., look for articles
published within the last 10 years and in English only) to
generate a more manageable or focused set of results.

Question 4

Q: How do you deal with finding too little papers?

Answer 4

A: consider broadening your search terms (e.g., the population of interest) to identify more articles. If you decide to refine your search terms by study design, keep the research question you are attempting to answer in mind.

Question 5

Q: What can a randomised controlled trial

(RCT) not tell you? But can help identify?

Answer 5

A randomized controlled trial (RCT) for example cannot tell you about how easy patients find adherence to a set of guidelines.

It could however help you to identify why a particular guideline is relevant to your patients care by assessing its effects within controlled situations, and therefore including such papers may be important.

Question 6

Q: Published information is often the only source of information utilised, but additional sources are also available. Name 4.

Answer 6

A: • Electronic search engines e.g., Medline, PubMed,
Sciencedirect
• Acknowledged experts;
• Practitioners;
•Theses, conference papers

Question 7

Q: As a general rule what is preferable to

avoid using when it comes to evidence/research?

Answer 7

A: internet-based information, unless this is obtained
from (and therefore can be referenced to) published sources, academic or government institutions, or charities for example.

Question 8

Q: A number of systems can be used to organise your review of research evidence. These can either be? (2)

Answer 8

A: generic systems or systems specifically focused on appraising evidence from specific study designs.

Question 9

Q: Irrespective of what you are reviewing, what are the two main features to keep in mind when finding and appraising evidence?

Answer 9

A: • What is the exposure I am interested in? This is key to finding and appraising any evidence you find. It should lead you to question how they quantified that exposure, whether the methods used are defined enough to be repeatable, as good as they could be (etc). It should also make you question whether the population that is sampled in the study is appropriate to learn best about this feature.
• What is the outcome I am interested in? Is this defined
consistently in all evidence I look at? Is the follow up time period appropriate? Remember that subjective
interpretation of data can really influence the quality of
results presented. Again, this will be affected by the
methods used and the population being studied (among other things).

Question 10

Q: What is critical appraisal?

Answer 10

A: The process of systematically examining research evidence to assess its validity, results and relevance before using it to inform a decision

(Keep up to date/Better communication with medics)

Question 11

Q: Why is critical appraisal an essential part of evidence-based clinical practice?

Answer 11

A: allows us to make sense of research evidence and begin to close the gaps between research and practice

Question 12

Q: Ideal design for outcome? (5)

Answer 12

A:  Therapeutic methods: efficacy, alternative methods etc. – RCT
 Diagnostic method: efficacy, reliability etc. – Cross sectional study
 Screening: value of tests which enable pre-symptomatic diagnosis – Cross sectional study
 Prognosis: Longitudinal cohort study
 Causation: environmental, lifestyle etc. factors and their impact on health – Cohort or case control study (or case reports)

Question 13

Q: What factors should be considered when doing critical appraisal? (9) +2

Answer 13

A: 	Question: 
	Design: 
	Population: 
	Methods: 
	Analysis: 
	Confounders: 
	Bias: 
	Ethics: 
	Interpretation: 

content and clarity.

Question 14

Q: Critical appraisal- question?

Answer 14

A: Is there a hypothesis? Is the question relevant?

Question 15

Q: Critical appraisal- design?

Answer 15

A: Is it cross-sectional, cohort, case-control, ecological, RCT? Hierarchy of studies. Is it appropriate?

Question 16

Q: Critical appraisal- population?

Answer 16

A: Sample size. Are results generalizable to other populations, inclusion criteria, subject recruiting

Question 17

Q: Critical appraisal- methods?

Answer 17

A: Exposure measurements, interview, measure, questionnaire etc.

What specific intervention was being considered and what was it being compared with?
What outcome was measured and how?
Duration of follow-up; Has the study continued for long enough to detect the effect of the intervention?

Question 18

Q: Critical appraisal- analysis?

Answer 18

A: Appropriate statistical tests. Chance? Confounding? taken into consideration?

Question 19

Q: Critical appraisal- confounders?

Answer 19

A: Presence of any confounders and attempts to exclude or adjust

Question 20

Q: Critical appraisal- bias?

Answer 20

A: Measurement/Selection? Randomised controlled trials? Non-randomised controlled studies (cohort & case control studies)? Completeness of follow-up; Is any assessment made of those who dropped out of the
study? Was the assessment blind (double blind)?

Question 21

Q: Critical appraisal- ethics?

Answer 21

A: Is the study ethical? Is informed consent obtained? Does other known research indicate that there is a reason why the study design should not have continued?

Question 22

Q: Critical appraisal- interpretation?

Answer 22

A: Do the authors interpret correctly? Do they make a causal inference? Bradford-Hill?

Question 23

Q: It is important that you summarise the paper first to get your own ideas about it clear. To do this, what 5 questions should be answered?

If you are reviewing a number of papers in one area, how is it often worth doing this?

Answer 23

A: • Why did they do it?
• What did they do?
• What did they find?
• What did they conclude?
• In your opinion, was the study conducted well?

within a series of tables

Question 24

Q: In what way is the use of evidence-based medicine (EBM) restricted? Therefore…

Answer 24

A: time and information resource restraints- do not expect to do this for every case

Question 25

Q: What does EBM provide?

Answer 25

A: an efficient and systematic way to locate the best evidence to answer clinical questions. Through simultaneous appraisal of that evidence (and inter-disciplinary collaboration), you can find the most relevant information to answer your question

Question 26

Q: Available evidence should be used in making patient care decisions, but evidence alone is not enough: it must be combined with what 2 things?

Answer 26

A: clinical judgement and patient preference