Diagnostic test accuracy Flashcards
Why are critical appraisal skills important?
- allows you to act as an independent professional
- allows you to inform opinions about which approaches and treatments are best for patients
- allows you to participate in the advancement of medical knowledge
Medical knowledge is always changing therefore it is important to critical appraise and not just rely on other’s opinions
What is a major problem with not evaluating evidence properly?
Causes the uptake of procedures/interventions by healthcare services that don’t have good quality evidence backing them
- approx. 50% of treatments in use are not based on good evidence
What are some examples of diagnostic tests?
Demographics - age, gender, ethnicity, social class Symptoms Questionnaires Physical signs Blood tests Imaging - X-ray, US Biopsy Post mortem
What makes a test helpful?
if it changes our ability to predict whether a person has a condition or not…. this is what guides treatment choice
The probability of the condition before the test should be altered by the test result
What factors need to be considered for taking up a new diagnostic test?
- How accurate is the test?
- Does having this test cause patient anxiety?
- how safe is this test?
- Will using this test benefit the patient?
- Would my colleague get the same result if he/she performed this test? - if a clinician is not confident
What proportion of medical tests are 100% accurate ?
about 1%
Why is it important to measure test accuracy?
Most tests make errors even if correctly performed
What does test accuracy mean?
how good a test is at telling us whether a pt has or doesn’t have a disease
Diagnosis is an essential preliminary test for knowing what treatment to carry out
What is test accuracy a comparison between?
A disease state estimated by a test of interest (index test) and the best estimate of the true disease state (reference standard)
Measurement of test accuracy is an explicit recognition that most tests make errors even if correctly performed
Define the index test and the reference test:
Index = test we are evaluating Reference = a more accurate test
What does blinded cross-classification mean?
compare results of index test and the reference test
- regardless of the results of the index test (+ve or -ve) the reference test needs to be compared to it - ideally blinded and independent
What are the important components of a test accuracy question?
Participants - presentation (severity, presence, duration symptoms), prior tests (has the patient been examined by a doctor before the index test is performed- more tests means fewer differential diagnoses the pt will have and therefore this will affect results)
Index test - conduct (experience of operator), technology (old vs new)
Target disorder (the condition we want the index test to identify)
Reference standard - most accurate method available fo detecting the target disorder
Can you have multiple index tests?
yes - the index test may be a new test or may be a test already used in practice that hasn’t been evaluated before its introduction
Index tests are assumed to be less accurate than the reference standard - ref standard represents the correct diagnosis against which the index tests are evaluated
What are some of the potential differences between primary and secondary care?
Technology can impact accuracy and this is likely to vary between primary and secondary care
Availability will also vary between primary and secondary care
Greater confidence and accuracy of interpreting results should be better in secondary care
What is the reference standard?
allows us to check if the index test results are in fact correct - it is how the outcome is measured
Cannot evaluate the accuracy of an index test without a reference standard
It is the most accurate way of determining if the target condition is present - so must be more accurate than the index test
it can comprise of more than one test
What are the 4 components that need to be considered when critically appraising the test accuracy of an index test?
1) internal validity (bias)
2) What are the results (numerical expression of test accuracy)
3) Can I apply the results to my patients (applicability)
4) What would be the impact of using the index test in my patients/population
What are the different types of bias?
spectrum bias
verification bias
review bias
What is spectrum bias?
are characteristics of the tested population clearly described - did the study avoid inappropriate exclusions
What is verification bias?
Did all participants get both index test and reference standard
What does review bias mean?
was the index test interpreted without knowledge of (blind to) the reference standard result and vice versa
What are 2x2 tables important for?
checking sensitivity and specificity of the test True +ves False -ves False +ves True -ves
What are summary measures?
specificity and sensitivity - need to understand these as they tell us about the performance of the test
What does sensitivity tell us?
tells us about the performance of the index test in individuals with the condition of interest (true positives)
What does specificity tell us ?
tells us about the performance of index test individuals without the condition of interest - true negatives
What are false negatives?
the number of pts that will be missed who have the condition
What are false positives?
the number of pts that wil be labelled as having the condition when they actually dont have the condition
What does high sensitivity mean?
higher the sensitivity, the lower the false negative rate - tells us about the usefulness of a test in ruling out a condition - a negative result is likely to be a true negative and therefore it is good at ruling out conditions
What does high specificity mean?
higher the specificity the lower the false positive rate - specificity tells us about the usefulness of a test in ruling in a condition
How is precision of results determined?
95% Confidence intervals
- 95% of the time the true result will be within these limits
- should have CI accompanying summary measures - provides worst and best case scenario
What does it mean by there is a trade off between summary measures?
trade off between sensitivity and specificity - as one increases the other decreases
What questions need to be considered when thinking about applicability?
Was there a clear study question
- is the study population spectrum similar
- can the index test be applied in your healthcare setting
- does the definition of the target condition match the condition you want to identify in practice
- also need to consider cost of the test, acceptability by both clinicians and pts
