Critical Appraisal Skills Flashcards
Describe the shift in clinical decision making
emphasis in clinical decision making shifted from “intuition, unsystematic clinical experience, and pathophysiologic rationale” to scientific, clinically relevant research
What are the criticisms of evidence based practice
Relies to heavily on research (not enough on patients)
Time consuming
Should not ignore patient preferences (or clinician’s judgement)
Large discrepancies in patient setting
Why is it important that we try to integrate evidence based medicine into clinical practice e
It allows clinicians to justify their decisions and explain to patients why they need to take their medication/treatment- thus improving patient adherence.
Describe the widespread use of evidence based practice
Nonetheless, used widely (in nursing, psychology, dentistry, speech pathology, psychology, social work, public health…)
What should we integrate into our decision-making according to Transdisciplinary Model of Evidence-based Practice
Populations, characteristics, needs, values and preferences.
Best Available evidence
Resources, including finances and the expertise of the practitioner.
What happens if resources are not available
Environmental and organisational context is outside the decision-making. If the resource is not available, then it cannot be considered in the decision-making context.
For example, if research ahs shown a treatment to be available, but it is too expensive, it is not available and so should not be considered. We need to take the evidence in the context of the patient.
What should you look at when finding the papers
Exposure?
Outcome?
Population?
What do we need to know when looking at papers
Study design?
Sampling?
Methods?
How do you determine whether the results answer your question
Can I interpret them?
What didn’t I know already?
Why is sampling important
The sampling needs to be correct to answer the question that we want to answer, and thus we need to know whether we are measuring the most accurate and most relevant outcomes.
What are the three key things to consider when conducting research
Ask focussed questions
Find the evidence
Critically appraise evidence
What is the purpose of the guidelines for each type of study, for example STROBE
They state what should be included- if something is missing, we have a reason to distrust the research.
Define critical appraisal
The process of systematically examining research evidence to assess its validity, results and relevance before using it to inform a decision
Essential part of evidence-based clinical practice, allowing us to make sense of research evidence and begin to close the gaps between research and practice
N.B., the quality of study designs is never certain, even if published
What does systematic mean in a research setting
Problem-solving
What do we need to consider in the methods
The validity and reliability of the method
What are the consequences of re-writes
Opportunity for errors to occur, and things not being reported.
What are the 5 characteristics of a good scientific method
Systematic – research process Logical – induction/deduction Empirical – evidence based Reductive – generalisation Replicable – methodology
Describe the process of deductive reasoning
Information – pattern – tentative hypothesis — theory
Describe inductive reasoning
Theory — hypothesis – observation — confirmation
Describe the uses of qualitative research
We can use qualitative research to form a theory, which can then be tested by empirical scientists.
Why should we be cautious about the frequency at which things are mentioned in research
Sometimes, the things that are rarely mentioned may have more importance than things that are mentioned frequently.
What is a key thing about scientific methodology
It is easy to follow and not open to interpretation.
Give an example of basic research
A study looking at how caffeine consumption impacts…
Does Caffeine Ingestion Improve Ca2+ binding with troponin?
Would this Facilitate Acto-Myosin Coupling?
Would this aid muscular contraction?
Looking at a cellular level and the mechanisms that explain the effects of an exposure.
Give an example of applied research
Does caffeine ingestion improve athletic performance?
Applied research assumes basic research and applies it to a more clinical setting.
What are some of the limitations of applied research
Confounding becomes more confusing- a lot of factors (that are not controlled) can interplay.
Practical difficulties in measuring the outcomes.
What do we need to know about basic research
You need to know about: your exposure your outcome your population group your expectations (as a scientist)
Describe the steps in the research process
Review available literature Formulate a question Select an appropriate research design Collect relevant data Interpret findings Publish findings
What do we need to consider at each stage of the research process
your exposure
your outcome
your population group
your expectations (as a scientist)
What do we need to break our question down into
Purpose, population and explanation (why is it important).
Where do we control for confounding
In the design, question and analysis- why critique is important in establishing the cause and effect association.
What should we consider in each stage of the research process
The Question: Recognise the need for new information - what question do you want to address?
Finding Evidence: If you have the wrong evidence, you’ll get the wrong answer to your question
Appraisal: Evaluate the validity and usefulness of the details (relative to your question)
Acting on evidence: what are you going to do with your new knowledge?
Evaluation and reflection: You made changes. Did it impact as you wanted it to?
What is the purpose of critical appraisal
Core part of all clinical practice
Enables you to find & make sense of, research evidence, and to put newly gained knowledge from appraisal of research into practice
Keep up to date
You want to know whether this is the ‘best’ test/ treatment for your patient
Describe our responsibilities as a scientist
All scientists have a responsibility to ensure that they conduct their work with honesty and integrity; to ensure that methods and results are reported in an accurate, orderly, timely and open fashion. …
What are the key things to consider when finding research evidence
Think back to your lectures on systematic reviews etc:
Make sure your question is solid
Structure your search comprehensively
Keep good notes!
Have questions that you want to ask of the papers you find
Papers are published to illustrate what the authors want you to know. Your questions are very likely to be different, and you have to read the articles, using your knowledge of the topic, methods used etc to learn from them how to answer your question…
How do we approach critical appraisal
Find models (be guided by others)
Problem formulation – which topic is under consideration and what are the constituent issues?
Literature search
Evaluation of findings (against your criteria/Q’s)
Analysis and interpretation of literature
What should we do in critical appraisal
Give a critical and evaluative account (not a description)
Summarise, synthesise and analyse
You should:
describe and analyse the existing evidence base
detail what gaps you’ve found
reveal similarities and differences, consistencies and inconsistencies and controversies
(try to) explain reasons for these…
Your methods will differ if you are reviewing one paper, or a topic area
What is the ideal study for therapeutic outcomes
Therapeutic methods: efficacy, alternative methods etc – RCT
What is the ideal study for diagnostic methods
Diagnosis method: efficacy, reliability etc – Cross sectional study
What is the ideal study for screening
Screening: value of tests which enable pre-symptomatic diagnosis – Cross sectional study
What is the ideal study for prognosis
Longitudinal cohort study
What is the ideal study for causation
Causation: environmental, lifestyle etc factors and their impact on health – Cohort or case control study (or case reports)
Describe the review checklists for each type of study
Experimental studies:
RCT: CONSORT
Infection control/intervention studies: ORION
Observational studies
STROBE
Genetic association studies: STREGA
Anecdotes of suspected drug adverse reactions: PHARMA
Tumour marker prognostic studies: REMARK
Internet e-Surveys: CHERRIES
Diagnostic accuracy studies: STARD
o PRISMA – Systematic Reviews and Meta-analyses.
o STARD – Reporting of diagnostic accuracy studies.
o MOOSE – Meta-analyses of observational studie
What do we need to consider in our discussion
Need to consider errors in our analysis and evaluate and reflect on what you have done and what others have done and why there may be differences.
Examines why findings sometimes contradictory
Highlights good practice (and things to avoid)
Considers what more needs to be done (and why)
What is key to reviewing what you are reading well
To review what you are reading well, you must understand the subject you are researching
What should we consider in critical appraisal
Why did they do it? What did they do? Was the design appropriate? Is the study original? Who is the study about? Was the study design sensible? What did they find? Is bias controlled for? Is the study blinded (if appropriate)? Were the appropriate statistics applied? What did they conclude?
Describe the key things to consider in the checklist
Question: Is there a hypothesis? Is the question relevant?
Design: Is it cross-sectional, cohort, case-control, ecological, RCT? Hierarchy of studies. Is it appropriate?
Population: Sample size. Are results generalisable to other populations
Methods: Exposure measurements, interview, measure, questionnaire etc.
Analysis: Appropriate statistical tests. Chance? Confounding?
Confounders: Presence of any confounders and attempts to exclude or adjust
Bias: Measurement/Selection?
Ethics: Is the study ethical?
Interpretation: Do the authors interpret correctly? Do they make a causal inference? Bradford-Hill?
Describe Cox regression
Considers whether the effect of a treatment under study has a multiplicative effect on the subject’s hazard rate (e.g., taking a statin may halve our immediate probability of having a MI)
What is meant by hazard ratio
the effect of an explanatory variable on the risk of an event
What do we need to be cautious about when practicing EBM
Use of EBM in practice is restricted by time and information resource restraints:
do not expect to do this for every case
when necessary, this is an extremely empowering skill which can help show your professionalism
credibility and value of clinical advice will be directly correlated with your ability to perfect such techniques
these skills are consistent with the code of ethics for all scientists
When do we need to intervene with EBM
We need to intervene and be vigilant when EBM doesn’t work in a clinical setting, it should not compromise clinical knowledge. We need clinical approval of the evidence in a case-by-case basis.
Why do we need to merge papers together
To explain what we see overall.
