Critical Appraisal Skills

Question 1

Describe the shift in clinical decision making

Answer 1

emphasis in clinical decision making shifted from “intuition, unsystematic clinical experience, and pathophysiologic rationale” to scientific, clinically relevant research

Question 2

What are the criticisms of evidence based practice

Answer 2

Relies to heavily on research (not enough on patients)
Time consuming
Should not ignore patient preferences (or clinician’s judgement)
Large discrepancies in patient setting

Question 3

Why is it important that we try to integrate evidence based medicine into clinical practice e

Answer 3

It allows clinicians to justify their decisions and explain to patients why they need to take their medication/treatment- thus improving patient adherence.

Question 4

Describe the widespread use of evidence based practice

Answer 4

Nonetheless, used widely (in nursing, psychology, dentistry, speech pathology, psychology, social work, public health…)

Question 5

What should we integrate into our decision-making according to Transdisciplinary Model of Evidence-based Practice

Answer 5

Populations, characteristics, needs, values and preferences.
Best Available evidence
Resources, including finances and the expertise of the practitioner.

Question 6

What happens if resources are not available

Answer 6

Environmental and organisational context is outside the decision-making. If the resource is not available, then it cannot be considered in the decision-making context.
For example, if research ahs shown a treatment to be available, but it is too expensive, it is not available and so should not be considered. We need to take the evidence in the context of the patient.

Question 7

What should you look at when finding the papers

Answer 7

Exposure?
Outcome?
Population?

Question 8

What do we need to know when looking at papers

Answer 8

Study design?
Sampling?
Methods?

Question 9

How do you determine whether the results answer your question

Answer 9

Can I interpret them?

What didn’t I know already?

Question 10

Why is sampling important

Answer 10

The sampling needs to be correct to answer the question that we want to answer, and thus we need to know whether we are measuring the most accurate and most relevant outcomes.

Question 11

What are the three key things to consider when conducting research

Answer 11

Ask focussed questions
Find the evidence
Critically appraise evidence

Question 12

What is the purpose of the guidelines for each type of study, for example STROBE

Answer 12

They state what should be included- if something is missing, we have a reason to distrust the research.

Question 13

Define critical appraisal

Answer 13

The process of systematically examining research evidence to assess its validity, results and relevance before using it to inform a decision
Essential part of evidence-based clinical practice, allowing us to make sense of research evidence and begin to close the gaps between research and practice
N.B., the quality of study designs is never certain, even if published

Question 14

What does systematic mean in a research setting

Answer 14

Problem-solving

Question 15

What do we need to consider in the methods

Answer 15

The validity and reliability of the method

Question 16

What are the consequences of re-writes

Answer 16

Opportunity for errors to occur, and things not being reported.

Question 17

What are the 5 characteristics of a good scientific method

Answer 17

Systematic – research process
Logical – induction/deduction
Empirical – evidence based
Reductive – generalisation
Replicable – methodology

Question 18

Describe the process of deductive reasoning

Answer 18

Information – pattern – tentative hypothesis — theory

Question 19

Describe inductive reasoning

Answer 19

Theory — hypothesis – observation — confirmation

Question 20

Describe the uses of qualitative research

Answer 20

We can use qualitative research to form a theory, which can then be tested by empirical scientists.

Question 21

Why should we be cautious about the frequency at which things are mentioned in research

Answer 21

Sometimes, the things that are rarely mentioned may have more importance than things that are mentioned frequently.

Question 22

What is a key thing about scientific methodology

Answer 22

It is easy to follow and not open to interpretation.

Question 23

Give an example of basic research

Answer 23

A study looking at how caffeine consumption impacts…
Does Caffeine Ingestion Improve Ca2+ binding with troponin?
Would this Facilitate Acto-Myosin Coupling?
Would this aid muscular contraction?
Looking at a cellular level and the mechanisms that explain the effects of an exposure.

Question 24

Give an example of applied research

Answer 24

Does caffeine ingestion improve athletic performance?

Applied research assumes basic research and applies it to a more clinical setting.

Question 25

What are some of the limitations of applied research

Answer 25

Confounding becomes more confusing- a lot of factors (that are not controlled) can interplay.
Practical difficulties in measuring the outcomes.

Question 26

What do we need to know about basic research

Answer 26

You need to know about: 
your exposure 
your outcome
your population group
your expectations (as a scientist)

Question 27

Describe the steps in the research process

Answer 27

Review available literature
Formulate a question
Select an appropriate research design
Collect relevant data
Interpret findings
Publish findings

Question 28

What do we need to consider at each stage of the research process

Answer 28

your exposure
your outcome
your population group
your expectations (as a scientist)

Question 29

What do we need to break our question down into

Answer 29

Purpose, population and explanation (why is it important).

Question 30

Where do we control for confounding

Answer 30

In the design, question and analysis- why critique is important in establishing the cause and effect association.

Question 31

What should we consider in each stage of the research process

Answer 31

The Question: Recognise the need for new information - what question do you want to address?
Finding Evidence: If you have the wrong evidence, you’ll get the wrong answer to your question
Appraisal: Evaluate the validity and usefulness of the details (relative to your question)
Acting on evidence: what are you going to do with your new knowledge?
Evaluation and reflection: You made changes. Did it impact as you wanted it to?

Question 32

What is the purpose of critical appraisal

Answer 32

Core part of all clinical practice
Enables you to find & make sense of, research evidence, and to put newly gained knowledge from appraisal of research into practice
Keep up to date
You want to know whether this is the ‘best’ test/ treatment for your patient

Question 33

Describe our responsibilities as a scientist

Answer 33

All scientists have a responsibility to ensure that they conduct their work with honesty and integrity; to ensure that methods and results are reported in an accurate, orderly, timely and open fashion. …

Question 34

What are the key things to consider when finding research evidence

Answer 34

Think back to your lectures on systematic reviews etc:
Make sure your question is solid
Structure your search comprehensively
Keep good notes!
Have questions that you want to ask of the papers you find
Papers are published to illustrate what the authors want you to know. Your questions are very likely to be different, and you have to read the articles, using your knowledge of the topic, methods used etc to learn from them how to answer your question…

Question 35

How do we approach critical appraisal

Answer 35

Find models (be guided by others)
Problem formulation – which topic is under consideration and what are the constituent issues?
Literature search
Evaluation of findings (against your criteria/Q’s)
Analysis and interpretation of literature

Question 36

What should we do in critical appraisal

Answer 36

Give a critical and evaluative account (not a description)
Summarise, synthesise and analyse
You should:
describe and analyse the existing evidence base
detail what gaps you’ve found
reveal similarities and differences, consistencies and inconsistencies and controversies
(try to) explain reasons for these…
Your methods will differ if you are reviewing one paper, or a topic area

Question 37

What is the ideal study for therapeutic outcomes

Answer 37

Therapeutic methods: efficacy, alternative methods etc – RCT

Question 38

What is the ideal study for diagnostic methods

Answer 38

Diagnosis method: efficacy, reliability etc – Cross sectional study

Question 39

What is the ideal study for screening

Answer 39

Screening: value of tests which enable pre-symptomatic diagnosis – Cross sectional study

Question 40

What is the ideal study for prognosis

Answer 40

Longitudinal cohort study

Question 41

What is the ideal study for causation

Answer 41

Causation: environmental, lifestyle etc factors and their impact on health – Cohort or case control study (or case reports)

Question 42

Describe the review checklists for each type of study

Answer 42

Experimental studies:
RCT: CONSORT
Infection control/intervention studies: ORION
Observational studies
STROBE
Genetic association studies: STREGA
Anecdotes of suspected drug adverse reactions: PHARMA
Tumour marker prognostic studies: REMARK
Internet e-Surveys: CHERRIES
Diagnostic accuracy studies: STARD
o PRISMA – Systematic Reviews and Meta-analyses.

o STARD – Reporting of diagnostic accuracy studies.
o MOOSE – Meta-analyses of observational studie

Question 43

What do we need to consider in our discussion

Answer 43

Need to consider errors in our analysis and evaluate and reflect on what you have done and what others have done and why there may be differences.
Examines why findings sometimes contradictory
Highlights good practice (and things to avoid)
Considers what more needs to be done (and why)

Question 44

What is key to reviewing what you are reading well

Answer 44

To review what you are reading well, you must understand the subject you are researching

Question 45

What should we consider in critical appraisal

Answer 45

Why did they do it?
What did they do?
Was the design appropriate?
Is the study original?
Who is the study about?
Was the study design sensible?
What did they find?
Is bias controlled for?
Is the study blinded (if appropriate)?
Were the appropriate statistics applied?
What did they conclude?

Question 46

Describe the key things to consider in the checklist

Answer 46

Question: Is there a hypothesis? Is the question relevant?
Design: Is it cross-sectional, cohort, case-control, ecological, RCT? Hierarchy of studies. Is it appropriate?
Population: Sample size. Are results generalisable to other populations
Methods: Exposure measurements, interview, measure, questionnaire etc.
Analysis: Appropriate statistical tests. Chance? Confounding?
Confounders: Presence of any confounders and attempts to exclude or adjust
Bias: Measurement/Selection?
Ethics: Is the study ethical?
Interpretation: Do the authors interpret correctly? Do they make a causal inference? Bradford-Hill?

Question 47

Describe Cox regression

Answer 47

Considers whether the effect of a treatment under study has a multiplicative effect on the subject’s hazard rate (e.g., taking a statin may halve our immediate probability of having a MI)

Question 48

What is meant by hazard ratio

Answer 48

the effect of an explanatory variable on the risk of an event

Question 49

What do we need to be cautious about when practicing EBM

Answer 49

Use of EBM in practice is restricted by time and information resource restraints:
do not expect to do this for every case
when necessary, this is an extremely empowering skill which can help show your professionalism
credibility and value of clinical advice will be directly correlated with your ability to perfect such techniques
these skills are consistent with the code of ethics for all scientists

Question 50

When do we need to intervene with EBM

Answer 50

We need to intervene and be vigilant when EBM doesn’t work in a clinical setting, it should not compromise clinical knowledge. We need clinical approval of the evidence in a case-by-case basis.

Question 51

Why do we need to merge papers together

Answer 51

To explain what we see overall.