Critical Appraisal Flashcards
Definition evidence based medicine
The conscientious, explicit and judicious use of current best evidence in making decisions about the care of the individual patient
5 steps of evidence based Medicine
- Question
(formulate clinical qu) - Evidence
(search for best evidence to answer qu) - Critically appraise evidence
(Validity, impact and applicability) - Application
(apply results to clinical practice, integrating critical appraisal with clinic expertise and patients views and circumstances) - Monitor
(Monitor the process, evaluating the effectiveness and efficacy of the whole process, and identifying ways of improving both for the future)
Definition critical appraisal
Assesses validity of the research and statistical techniques employed in studies, and generated clinically useful information from them.
It seems to answer the 2 major qus:
- Does the research have internal validity?
- Does the research have external validity?
Internal validity definition
To what extent a study measures what it set out to measure.
How good are the research methods used by the researchers to answer the clinical qu
(Efficacy)
External validity definition
To what extent can the results from the study be generalised to a wider population? Will the results be the same in real life settings?
(Effectiveness)
Efficacy definition
The impact of interventions under optimal (trial) conditions
Internal validity
Effectiveness definition
Describes whether the interventions have the intended or expected effect under ordinary (clinical) circumstances
(External validity / generalisability)
Structure for formulating a clinical question
PICO
Patient / problem
Intervention
Comparison (not always needed)
Outcome
What is a peer reviewed journal?
A publication that requires each submitted article to be independently examined by a panel of experts, who are non-editorial staff of the journal.
Majority of peers need to approve to be considered for publication.
Anonymous process (can be double blinded).
Benefits of peer review
- Forces authors to meet certain standards laid down by researchers and experts in that field.
- More likely that mistakes or flaws detected
I.e quality assurance.
Therefore peer reviewed journals generally held in greater esteem
Disadvantages of peer review process
- Delay between submission and publication
- Peer reviewers might guess identity of author(s)
- Revolutionary or unpopular conclusions can face opposition leading to preservation of the status quo.
- Still doesn’t guarantee lack of errors or fraudulent research
The primary hypothesis
The clinical question is formalised into a position statement that will be proven true or false in the experiment. Generated before data collection.
Same or closely related to the clinical question.
NB
Not all studies designed to test hypothesis. Some studies (case reports or qualitative) can be used to generate hypotheses
What is a confounder?
A confounder has a triangular relationship with both the exposure and the outcome, but most importantly, it is not on the causal pathway.
It makes it appear as if there is a direct relationship between exposure and the outcome or it might even mask an association that would otherwise have been present.
Can cause overestimation or underestimation of the true association and Can even change the direction of the observed effect.
What is a positive confounder?
Results in an association between two variables that are not associated.
What is a negative confounder?
Masks an association that is really present
How would a good study deal with confounders?
- accept possibility that confounding May be an issue
- list confounding factors and explain their impact on any relationship under investigation
- describe how confounding factors were controlled at the design and analysis stage
Dealing with confounders
Once identified there are a number of techniques that can be used to deal with confounding either at design stage or analysis stage:
- eliminate confounding factors altogether (exclusion criteria)
- nullify their effects by spreading them equally between the different groups of the study (randomisation)
- account for their effects on the results using statistical techniques (eg regression)
Difference between bias and confounding
Bias leads to the wrong results
Confounding leads to the wrong conclusion
Confounding differs from bias in that confounding is not caused by a mistake made by the researchers, it arises from a real life relationship that already exists between exposures and outcomes under consideration
Bias definition
- bias leads to the wrong results
- if a study suffers bias the results data are incorrect due to the bias
Ie the conclusion is correct judged solely on collected data but if there had been no bias different results would have been generated
Confounding definition
Results data are correct but the interpretation of the results is wrong leading to the wrong conclusion. Happens when researchers don’t appreciate the entire situation.
Types of bias
- selection bias
- performance bias
- detection bias
- attrition bias
- reporting bias
Selection bias
Problem with recruiting and allocating subjects.
Deal with this by random allocation and concealment of allocation
Performance bias
Problem with way data gathered in study, such results have been unduly influenced by the expectations of researchers and subjects.
To deal with this: blinding and placebo arms
Detection bias
Can be minimised by blinding
Attrition bias
It is important to see whether the intention to treat or the per protocol approach was used because this may present a biased result
Reporting bias
Outcome reporting bias, publication bias
In relation to bias a good study will
Strive to use method to minimise risk of bias
Acknowledge the presence of bias when it occurs and discuss how results may have been influenced
Suggest how future researchers should modify their methods to reduce risk of bias if possible
With regards to population and sampling a good study will
- define target population
- describe sampling method
- list inclusion criteria and relate them to the aims of the study
- list and explain exclusion criteria
- illustrate how many people were included and excluded, often in the form of a flowchart or table
- explain how sample size determined
- comment on how the sample population represents the target population
In relation to selection bias a good study will….
Allow sufficient time and resources to recruit subjects without resorting to
Shortcuts
Explain how selection process minimised selection bias
Acknowledge mistakes or methodological compromises that were made
Make suggestions on how mistakes can be avoided in future research if possible
3 key concepts of critical appraisal
- Validity
- Trustworthiness of results
- Value and relevance
Validity of study
Rigour of study and extent to which the conclusions of research are true in context in which restate was undertaken (internal validity).
A more valid:
- Design
- Method
- Procedure
Produces less bias results giving us more reliable generalisable conclusions
