Critical appraisal Flashcards
What 4 factors are involved when making an evidence based decision?
1) Evidence from research
2) Clinical expertise
3) Available resources
4) Patient preferences
State 3 reasons why we need evidence based medicine.
1) Increases medical knowledge
2) There is limited time to read
3) Traditional resources such as books can prove to be inadequate (are often out of date)
4) There is a disparity between diagnostic skills/ clinical judgement and up to date knowledge.
What is evidence based medicine?
A process for identifying and using the most up to date and relevant evidence to inform decisions for individual patient problems.
Give 3 reasons why evidence based medicine is possible.
1) The development of strategies for effectively identifying and appraising evidence.
2) The creation of systematic reviews and summaries
3) The creation of evidence-based journals and clinical guidelines
4) The creation of information systems
5) The identification and application of strategies for lifelong learning.
What are background questions?
Background questions are informational questions that improve the understanding of a topic.
The questions usually have one or two concepts and do not require the process used in formulating a research or evidence based decision-making questions.
They basically ask general knowledge about a disorder.
E.G. What causes breast cancer?
What are foreground questions?
Foreground questions ask for specific knowledge to inform clinical decisions.
These questions typically concern a specific patient or particular population.
Foreground questions tend to be more specific and complex compared to background questions.
E.G. In younger women with breast cancer, is mastectomy with chemotherapy more effective than mastectomy done alone, in reducing the risk of cancer recurrence?
What are typically the 4 essential components to a foreground question?
1) Patient, problem, population
2) Intervention
3) Comparison intervention/ control
4) Clinical outcomes
When asking foreground questions, accuracy can be classed as what?
Having a low false positive rate and a low false negative rate.
What are the aspects of a PICO question?
- Patient/ population/ problem
- Intervention
- Comparator/ control
- Outcome
What are the aspects of a PEO question?
- Population/ patient/ problem
- Exposure
- Outcome
When are PEO questions used over PICO questions?
When there is no obvious control group.
Which studies are used for the following question types?
1) Diagnosis
2) Aetiology/ harm
3) Prognosis
4) Treatment
5) Evaluation
1) Cross sectional analytical study/ SR
2) Cohort study/ population based case-control study/ SR
3) Cohort study/SR
4) RCT/ SR of RCTs
5) Qualitative research/ SR/ meta-analysis
What is the difference between case-control and cohort studies?
Case control studies are retrospective. Cohort studies follow people prospectively.
Describe 3 benefits of systematic reviews over individual primary studies.
1) Include all available research evidence to answer a question.
2) Include research that is unpublished or has been published in non-English language journals.
3) Increase the total sample size (increases levels of certainty and precision).
4) Indicate heterogeneity (variation) among studies
5) Permit sub-group analyses
6) Permit sensitivity analysis.
What is bias?
The systematic introduction of error into a study that can distort the results in a non-random way.
What are the 3 discrete steps in appraising some evidence?
1) Are the results of the study valid?
2) What are the results?
3) Can I apply the results to this patient’s care?
What 5 things must you assess a research study for?
1) Bias
2) Applicability
3) Limits
4) Value
5) Whether the evidence provided is useful for patient care.
In terms of results, what must you report for studies looking at each of the following factors?
1) Therapy (RCT)
2) Diagnosis
3) Prognosis
4) Aetiology/ harm
1) Relative risk reduction, absolute risk reduction, odds ration, NNT, confidence intervals.
2) Sensitivity, specific, PPV, NPV, likelihood ratios.
3) Likelihood of outcomes overtime, how precise prognosis estimates are (relative risk/ odds ratio).
4) Relative risk, odds ration, NNtH.
What must you consider when deciding if you can use the results of a study?
GENERALISABILITY:
1) How similar are the patients in the study to your patient?
2) Can the local health service provide the intervention/ diagnostic test?
3) What are the benefits and costs (potential harms/ financial costs)?
For systematic reviews, state what must be considered for each of the following questions:
1) Did the review ask a clearly focused question?
2) Did the review include the right type of study?
1) Consider if the question is focused in terms of: the population studied, the intervention given/ the exposure and the outcomes considered.
2) Consider if the included studies: address the review’s question and have an appropriate study design.
For systematic reviews, state what must be considered for each of the following questions:
1) Did the reviewers try to identify all relevant studies?
Consider:
- which bibliographic databases were used.
- if there was a follow-up from the reference list.
- if there was personal contact with experts.
- if the reviewers searched for unpublished studies.
- if the reviewers searched for non-English language studies.
**Increases the number of studies and ensures all relevant studies (incl. those with null or negative results) are included in the review.
For systematic reviews, state what must be considered for each of the following questions:
1) Did the reviewers assess the quality of the included studies?
Consider:
- if a clear, pre-determined strategy was used to determine which studies were included.
- a scoring system
- more than one assessor.
**Look for individual studies assessed for quality by more than one reviewer, using standard criteria. This means that it is more likely that only studies of good quality are included.
For systematic reviews, state what must be considered for each of the following questions:
1) If the results of the studies have been combined, was it reasonable to do so?
Consider whether:
- the results of each study are clearly displayed.
- the results were similar from study to study.
- the reasons for any variations in results are discussed.
**If studies are very dissimilar, then results should not be combined. Combined results provide stronger evidence, but only when it is appropriate to do so.
For systematic reviews, state what must be considered for each of the following questions:
1) How are the results presented and what is the main result?
Consider:
- how the results are expressed (odds ratio/ relative risk, etc)
- how large this size of result is and how meaningful it is.
- how you would sum up the bottom-line result in one sentence.
For systematic reviews, state what must be considered for each of the following questions:
1) How precise are these results?
Consider:
- if a confidence interval were reported.
- would your decision about whether or not to use this intervention be the same at the upper confidence limit as at the lower confidence limit?
- if a p-value is reported where confidence intervals are unavailable.
For systematic reviews, state what must be considered for each of the following questions:
1) Can the results be applied to the local population?
Consider whether:
- the population sample covered by the review could be different from your population in ways that would produce a different result.
- your local setting differs much from that of the review.
- you can provide the same intervention in your setting.
**Because SRs can use heterogenous study populations, there may be details on sub-groups.
For systematic reviews, state what must be considered for each of the following questions:
1) Were all important outcomes considered?
2) Should policy or practice change as a result of the evidence contained in this review?
1) Consider outcomes from the POV of the individual, policy makers and professionals, family/ carers and the wider community.
2) Consider whether any benefit reported outweighs any harm and/or cost. If this information is not reported, can it be filled in from elsewhere?
For randomised control trials, state what must be considered for each of the following questions:
1) Did the study ask a clearly focused question?
2) Was this a randomised control trial and was it appropriately so?
1) Consider if the question is focused in terms of: population studied, intervention given and outcomes considered.
2) Consider why this study was carried out as an RCT and if this was the right research approach for the question being asked.
For randomised control trials, state what must be considered for each of the following questions:
1) Were participants appropriately allocated to intervention and control groups?
Consider:
- how participants were allocated to intervention and control groups. Was the process truly random?
- whether the method of allocation was described. Was a method used to balance the randomisation, e.g. stratification?
- how the randomisation schedule was generated and how a participant was allocated to a study group
- if the groups were well balanced. Are any differences between the groups at entry to the trial reported?
- if there were differences reported that might have explained any outcome(s) (confounding)
For randomised control trials, state what must be considered for each of the following questions:
1) Were participants, staff and study personnel ‘blind’ to participants’ study group?
Consider:
- the fact that blinding is not always possible
- if every effort was made to achieve blinding
- if you think it matters in this study
- the fact that we are looking for ‘observer bias’
For randomised control trials, state what must be considered for each of the following questions:
1) Were all of the participants who entered the trial accounted for at its conclusion?
Consider:
- if any intervention-group participants got a control group option or vice versa.
- if all participants were followed up in each study group (was there loss-to-follow-up?)
- if all the participants’ outcomes were analysed by the groups to which they are originally allocated (intention to treat analysis).
- what additional information would you liked to have seen to make you feel better about this.
For randomised control trials, state what must be considered for each of the following questions:
1) Were the participants in all groups followed up and data collected in the same way?
2) Did the study have enough participants to minimise the play of chance?
1) if, for example, they were reviewed at the same time intervals and if they received the same amount of attention from researchers and health workers. Any differences may introduce performance bias.
2) if there is a power calculation. This will estimate how many participants are needed to be reasonably sure of finding something important (if it really exists and for a given level of uncertainty about the final result).
For randomised control trials, state what must be considered for each of the following questions:
1) How are the results presented and what is the main result?
Consider:
1) if, for example, the results are presented as a proportion of people experiencing an outcome, such as risks, or as a measurement, such as mean or median differences, or as survival curves and hazards
2) how large this size of result is and how meaningful it is
3) how you would sum up the bottom-line result of the trial in one sentence
For randomised control trials, state what must be considered for each of the following questions:
1) How precise are the results?
Consider:
1) if the result is precise enough to make a decision
2) if a confidence interval were reported. Would your decision about whether or not to use this intervention be the same at the upper confidence limit as at the lower confidence limit?
3) if a p-value is reported where confidence intervals are unavailable
For randomised control trials, state what must be considered for each of the following questions:
1) Were all important outcomes considered so the results can be applied?
Consider whether:
1) the people included in the trial could be different from your population in ways that would produce different results
2) your local setting differs much from that of the trial
3) you can provide the same treatment in your setting
Consider outcomes from the point of view of the:
1) Individual
2) policy makers and professionals
3) family/ carers
4) wider community
Consider whether:
1) any benefit reported outweighs any harm and/or cost. If this information is not reported can it be filled in from elsewhere?
2) policy or practice should change as a result of the evidence contained in this trial
What is meant by the term ‘statistical power’?
Statistical power, or the power of a hypothesis test is the probability that the test correctly rejects the null hypothesis. That is, the probability of a true positive result. It is only useful when the null hypothesis is rejected.