Critical Appraisal Jan Flashcards
What is the framework for an answerable question?
PICO
P= patient/population
I= intervention
C= control
O= outcome
What to search for with therapy questions?
What is the best treatment taking into account benefits and harm
What to search for with diagnostic questions?
What is the best test? How can we tell if it’s wrong?
Best study design for diagnostic question?
Cross sectional study which analyses data at a single point in time.
What to search for with prognostic questions?
What is going to happen if…
Best study design for prognosis?
Cohort
What to search for with aetiology questions?
What has caused this to happen?
Best study design for aetiology?
Case control
Cohort study
What to search for with evaluation/acceptance questions?
What do patients think about it?
Best study design for evaluation?
Survey
What are the benefits of a systematic review?
Larger sample size
Includes all evidence related to question, including unpublished.
Indicates need for further research and can do sub group analyses.
What is bias?
Systematic introduction of error into study that affects study in non-random way.
How should a study be assessed?
Bias
Applicability
Limits
Overall value of research
Why is critical appraisal important?
Evidence for effective patient care.
What are the principal steps of critical appraisal?
What are the results?
Are the results valid?
Are they applicable to patient care?
-> Study MUST use a clearly focused question and right type of study for question
What is hand searching in research?
Searching for medical journals not indexed in Medline or Embase
Why is a systematic review not beneficial for some studies?
Provides an average for a larger population, not useful for patient subgroups.
What are the important areas that a critically relevant study has?
Clearly focused question
Uses right study design
Attempts to identify all relevant study with a search strategy
Assessment of study quality
Main result and how its presented (eg odds ratio)
Precision of results (using p value)
Applicability
Considers all outcomes
Should the polcity be changed?
Why is a search strategy important for critical appraisal?
You may find important studies that are untranslated to English. Attempts to contact author are important. It is also ideal for excluding irrelevant studies
How to organise a search on a database?
Identify key concepts in the hypothesis to use as a search term, and finding their related synonyms to find as many relevant articles as possible.
E.g Does physical activity in the elderly delay dementia?
Key concepts will be:
Physical activity so synonyms include running, walking and exercis*
Elderly so synonyms include geriatric, old,
Dementia so synonyms include Alzhiemer’s
How to search for phrases on a database?
Quotations should be put around the search term.
What is the first important step when doing a search?
Unticking ‘Map Term to Subject Heading.’
When is ‘OR’ used in searching?
Used for linking a group of related terms when searching on database, e.g exercis* OR walking OR running.
When is ‘AND’ used in searching?
Combining the separate search concepts to find relevant articles mentioning all.
What is a background question?
Questions to gain knowledge about the pathology of the condition.
What is a foreground question?
Questions to gain knowledge of how to manage a patient’s condition, for diagnosis, prognosis and which treatment is most effective and the outcomes. This can be searched by framing a foreground question using the PICO framework.
How is a systematic review critically appraised for its subject?
Whether:
All relevant studies have been identified
The search strategy is documented
Inclusion of the right studies which address the focused question
How is a systematic review critically appraised for outcomes?
Based on:
Reviewers being blind to the authors
Quality of the accessed studies
Homogeneity of the included studies
Presentation of main outcomes
Weaknesses and strengths of the included studies
How is an RCT critically appraised for the subject?
Whether:
—> Random allocation occured in the cohort
—> Assessment of the validity of each included trial
—> The similarity between the groups (should be as similar as possible) which should be in the baseline characteristics section for variables that affect the outcome
—>The inclusion/exclusion criteria for the study
—> The baseline data for assessing intervention effectiveness can be compared
How is an RCT critically appraised for the study duration?
Whether:
Participants, clinicians and reviewers were blinded to allocation
If Follow up was long enough to measure effects
Proportion lost to follow up
Was the study complete?
How is an RCT critically appraised for outcomes?
Based on:
Intention-to-treat analysis
An objective independent assessment was carried out
How is a cohort study critically appraised?
Whether:
Cohort is representative of population
Both groups are comparable
Confounding factors are managed via techniques such as stratification
How is a cohort study critically appraised in the study duration?
Whether:
Participants/assessors were blinded
If the duration of the study was long enough to measure disease
Proportion lost to follow up
How is a cohort study critically appraised for its outcomes?
Based on:
Detection of confounding factors
Outcome is accurately measured using odds ratio or another objective measurement
Objective independent assessment of the study
How is a case control study critically appraised for its subject?
Whether:
There is random selection of the control
Groups are comparable
Addresses a focused queston
Confounding factors are managed via techniques such as stratification
How is a case control study critically appraised for its study duration?
Whether:
Blinding occurs of participants and assessors
Duration is long enough to assess exposures
How is a case control critically appraised for outcomes?
Based on:
Cases and controls being assessed using the same method
All selected participants are included in the analysis
There is an independent objective assessment
Measurement of effect estimate using relative risk ratio
What can bias the results of an RCT?
Attrition bias between the two groups, where either the control or exposure group has unequal loss of participants.
What is the ideal loss to follow up number?
Maximum of 20%.
When can blinding not be performed for an RCT?
For safety reasons, such as the use of warfarin.
What is a dichotomous outcome?
Answers fall into a category of ‘yes’ or ‘no’
What is a continuous outcome?
Answers with a range of values.
What is an important feature that should be included in a critical appraisal of observational studies?
Observational studies include cohort and case-control studies, which must have confounding factors identified and using techniques to mitigate their effect, due to their influence on the outcome in the groups.
The measurement of exposures should also be identified.
How can confounding factors be reduced?
Balancing the confounding factors across groups with:
Restriction
Stratification
Matching
Multiple regression techniques
What type of bias is caused by loss to follow-up?
Follow-up within a study is keeping track of all participants in order to prevent attrition bias and measurement error. This is the difference between the measured value and the true value, which increases the risk of a Type 2 statistical error.
—> Short follow up has a risk of association not being measured. A long follow up does not affect the detection of associations but it is costly and unethical collection of scientific info with no value.
What is a type 1 statistical error?
Rejection of the null hypothesis when the null hypothesis is true.
What is a Type 2 statistical error?
Accepting the null hypothesis when the null hypothesis is wrong.
How can we determine the strength between outcome and association?
Using measures of effect like relative risk or odds ratio, where a value less than 1.5 is weak.
Which value indicates the precision of the effect estimate?
Confidence intervals, a range of values where it is likely that the mean lies within.
What is intention-to-treat analysis?
All participants are evaluated based on the original group they were randomised into, which maintains the balancing for factors across groups and allows the results to be more generalisable because patients do not always comply with treatment plans.
What is per-protocol analysis?
Only participants who followed the protocol exactly are evaluated for effect outcome. The issue with this is a risk of overestimating effects and should not be the only analysis form used.
What is as-treated analysis?
Regardless of original group allocation, participants are evaluated based on the actual treatment/group they align most with, which is useful for measuring intervention side effects
