Diagnostic test accuracy

Question 1

Why are critical appraisal skills important?

Answer 1

• allows you to act as an independent professional
• allows you to inform opinions about which approaches and treatments are best for patients
• allows you to participate in the advancement of medical knowledge

Medical knowledge is always changing therefore it is important to critical appraise and not just rely on other’s opinions

Question 2

What is a major problem with not evaluating evidence properly?

Answer 2

Causes the uptake of procedures/interventions by healthcare services that don’t have good quality evidence backing them
- approx. 50% of treatments in use are not based on good evidence

Question 3

What are some examples of diagnostic tests?

Answer 3

Demographics - age, gender, ethnicity, social class
Symptoms
Questionnaires
Physical signs 
Blood tests 
Imaging - X-ray, US
Biopsy 
Post mortem

Question 4

What makes a test helpful?

Answer 4

if it changes our ability to predict whether a person has a condition or not…. this is what guides treatment choice

The probability of the condition before the test should be altered by the test result

Question 5

What factors need to be considered for taking up a new diagnostic test?

Answer 5

• How accurate is the test?
• Does having this test cause patient anxiety?
• how safe is this test?
• Will using this test benefit the patient?
• Would my colleague get the same result if he/she performed this test? - if a clinician is not confident

Question 6

What proportion of medical tests are 100% accurate ?

Answer 6

about 1%

Question 7

Why is it important to measure test accuracy?

Answer 7

Most tests make errors even if correctly performed

Question 8

What does test accuracy mean?

Answer 8

how good a test is at telling us whether a pt has or doesn’t have a disease
Diagnosis is an essential preliminary test for knowing what treatment to carry out

Question 9

What is test accuracy a comparison between?

Answer 9

A disease state estimated by a test of interest (index test) and the best estimate of the true disease state (reference standard)

Measurement of test accuracy is an explicit recognition that most tests make errors even if correctly performed

Question 10

Define the index test and the reference test:

Answer 10

Index = test we are evaluating 
Reference = a more accurate test

Question 11

What does blinded cross-classification mean?

Answer 11

compare results of index test and the reference test
- regardless of the results of the index test (+ve or -ve) the reference test needs to be compared to it - ideally blinded and independent

Question 12

What are the important components of a test accuracy question?

Answer 12

Participants - presentation (severity, presence, duration symptoms), prior tests (has the patient been examined by a doctor before the index test is performed- more tests means fewer differential diagnoses the pt will have and therefore this will affect results)

Index test - conduct (experience of operator), technology (old vs new)

Target disorder (the condition we want the index test to identify)

Reference standard - most accurate method available fo detecting the target disorder

Question 13

Can you have multiple index tests?

Answer 13

yes - the index test may be a new test or may be a test already used in practice that hasn’t been evaluated before its introduction

Index tests are assumed to be less accurate than the reference standard - ref standard represents the correct diagnosis against which the index tests are evaluated

Question 14

What are some of the potential differences between primary and secondary care?

Answer 14

Technology can impact accuracy and this is likely to vary between primary and secondary care
Availability will also vary between primary and secondary care
Greater confidence and accuracy of interpreting results should be better in secondary care

Question 15

What is the reference standard?

Answer 15

allows us to check if the index test results are in fact correct - it is how the outcome is measured
Cannot evaluate the accuracy of an index test without a reference standard
It is the most accurate way of determining if the target condition is present - so must be more accurate than the index test
it can comprise of more than one test

Question 16

What are the 4 components that need to be considered when critically appraising the test accuracy of an index test?

Answer 16

1) internal validity (bias)
2) What are the results (numerical expression of test accuracy)
3) Can I apply the results to my patients (applicability)
4) What would be the impact of using the index test in my patients/population

Question 17

What are the different types of bias?

Answer 17

spectrum bias
verification bias
review bias

Question 18

What is spectrum bias?

Answer 18

are characteristics of the tested population clearly described - did the study avoid inappropriate exclusions

Question 19

What is verification bias?

Answer 19

Did all participants get both index test and reference standard

Question 20

What does review bias mean?

Answer 20

was the index test interpreted without knowledge of (blind to) the reference standard result and vice versa

Question 21

What are 2x2 tables important for?

Answer 21

checking sensitivity and specificity of the test 
True +ves 
False -ves
False +ves 
True -ves

Question 22

What are summary measures?

Answer 22

specificity and sensitivity - need to understand these as they tell us about the performance of the test

Question 23

What does sensitivity tell us?

Answer 23

tells us about the performance of the index test in individuals with the condition of interest (true positives)

Question 24

What does specificity tell us ?

Answer 24

tells us about the performance of index test individuals without the condition of interest - true negatives

Question 25

What are false negatives?

Answer 25

the number of pts that will be missed who have the condition

Question 26

What are false positives?

Answer 26

the number of pts that wil be labelled as having the condition when they actually dont have the condition

Question 27

What does high sensitivity mean?

Answer 27

higher the sensitivity, the lower the false negative rate - tells us about the usefulness of a test in ruling out a condition - a negative result is likely to be a true negative and therefore it is good at ruling out conditions

Question 28

What does high specificity mean?

Answer 28

higher the specificity the lower the false positive rate - specificity tells us about the usefulness of a test in ruling in a condition

Question 29

How is precision of results determined?

Answer 29

95% Confidence intervals

• 95% of the time the true result will be within these limits
• should have CI accompanying summary measures - provides worst and best case scenario

Question 30

What does it mean by there is a trade off between summary measures?

Answer 30

trade off between sensitivity and specificity - as one increases the other decreases

Question 31

What questions need to be considered when thinking about applicability?

Answer 31

Was there a clear study question

• is the study population spectrum similar
• can the index test be applied in your healthcare setting
• does the definition of the target condition match the condition you want to identify in practice
• also need to consider cost of the test, acceptability by both clinicians and pts