Diagnostic/Screening Test Evaluation Flashcards
What is a test? What are some examples?
process or device designed to detect clinical signs, substance/agent, tissue change, or body response
physical exam, hematology, serology, biochemistry, histopathology
What is the difference between accuracy and precision?
ACCURACY = the ability of a test to come to the true value (“hitting the bullseye”)
PRECISION = ability of a test to give repeatable results
What is analytical sensitivity and specificity?
SENSITIVITY = lowest concentration the test can detect, limit of detection
SPECIFICITY = degree of cross-reactivity with non-target agents; high specificity = only detect target agent
(all in a lab environment)
What are the requirements of diagnostic test evaluation?
- test will detect diseased animals correctly
- test will detect non-diseased animals correctly
What is diagnostic sensitivity and specificity?
SENSITIVITY = probability of a positive test given that the animal is diseased
SPECIFICITY = probability of a negative test, given the animal is non-diseased
What is a gold standard?
test or procedure that is absolutely accurate (still not perfect) - as close as we can get
- histopathological and microbiological examination of small intestine for Johne’s
- immunofluorescence antibody test for rabies
What 4 components are typically needed to properly diagnose disease?
- identification of agent - culture, PCR, molecular confirmation, antigen
- histological changes consistent with disease
- presence of specific antibodies
- clinical signs of exposure to agent
Where do healthy (non-diseased) animals often come from?
naive populations free from certain agents
Sensitivity and specificity is not often reported by manufacturers of diagnostic tests. How are they recorded?
independent studies report values in certain populations —> further determined by carrying out specially designed studies from OIE guidelines
How is the definition of diseased and non-diseased determined in diagnostic tests with continuous scales?
determination of cut-off values to assign +ve and -ve status
(continuous scales - ELISA optical density, glucose, ALT, ALP, creatinine, BUN, cell counts)
When do tests on a continuous scale develop false positives?
cutoff miscalculates healthy animals as diseased
When do tests on a continuous scale develop false negatives?
cutoff miscalculates diseased animals as healthy
When are tests on a continuous scale considered a gold standard?
cutoff does not miscalculate diseased and healthy anmals
What is the sensitivity and specificity of this test like?
not very accurate with a lot of overlap for false positives and negatives - may not be worth time or money
What is the classic presentation of diagnostic test values? How is sensitivity and specificity calculated?
2x2 tables
a = true pos
b = false pos
c = false neg
d = true neg
How is accuracy calculated based on 2x2 tables?
diagonal
What is apparent prevalence? How is it used to estimate true prevalence?
estimating disease prevalence with an imperfect test - what seems to be (not actually happening in real life)
use sensitivity and specificity of the test
Lou is a 5-year-old Catahoula Leopard intact male dog from Louisiana. He was tested for Dirofilaria immitis using the IDEXX SNAP test with 85.8% accuracy, 84.1% sensitivity, and 96.9% specificity. If he tests positive, how confident are we that Lou is truly infected and needs to be treated? If he tests negative, how confident are we that Lou does not need to be treated?
- prevalence of heartworm in Louisiana is 5.78%
PPV = p(D+|T+) = a/(a+b) = 49/79 = 62%
- given that Lou was positive, there is a 62% probability that he has at least one female heartworm
NPV = p(D-|T-) = d/(c+d) = 912/921 = 99%
- given that Lou was negative, there is a 99% probability that he truly doesn’t have heartworms
(makes sense - low prevalence = high NPV and low PPV)
What is the sensitivity, specificity, and accuracy of the test in this 2x2 table?
What is an important note about conditional probabilities?
p(A|B) =/= p(B|A)
What do predictive values depend on?
PREVALENCE - study values CANNOT be used for patients (studies in other countries or states are not the same in Louisiana regarding Lou)
How do predictive values differ with low prevalence and high prevalence?
LOW PREVALENCE:
- PPV decreases: less accurate
- NPV increases: rare disease upon negative result, almost 100% sure animal is not infected
HIGH PREVALENCE:
- PPV increases: good accuracy of positive result
- NPV decreases
A practice is using an FeLV test with a sensitivity of 90% and a specificity of 95%. Assuming the prevalence of feline leukemia in the area is 5%, what is the negative predictive value of the test?
140 wallabies are serologically tested for disease. 35 test seropositive and 105 test seronegative. However, postmortem data reveals 5/35 of the seropositive wallabies are disease-free and 4/105 of the seronegative are diseased. What is the positive predictive value of the serological test?