Diagnostic/Screening Test Evaluation Flashcards
What is a test? What are some examples?
process or device designed to detect clinical signs, substance/agent, tissue change, or body response
physical exam, hematology, serology, biochemistry, histopathology
What is the difference between accuracy and precision?
ACCURACY = the ability of a test to come to the true value (“hitting the bullseye”)
PRECISION = ability of a test to give repeatable results
What is analytical sensitivity and specificity?
SENSITIVITY = lowest concentration the test can detect, limit of detection
SPECIFICITY = degree of cross-reactivity with non-target agents; high specificity = only detect target agent
(all in a lab environment)
What are the requirements of diagnostic test evaluation?
- test will detect diseased animals correctly
- test will detect non-diseased animals correctly
What is diagnostic sensitivity and specificity?
SENSITIVITY = probability of a positive test given that the animal is diseased
SPECIFICITY = probability of a negative test, given the animal is non-diseased
What is a gold standard?
test or procedure that is absolutely accurate (still not perfect) - as close as we can get
- histopathological and microbiological examination of small intestine for Johne’s
- immunofluorescence antibody test for rabies
What 4 components are typically needed to properly diagnose disease?
- identification of agent - culture, PCR, molecular confirmation, antigen
- histological changes consistent with disease
- presence of specific antibodies
- clinical signs of exposure to agent
Where do healthy (non-diseased) animals often come from?
naive populations free from certain agents
Sensitivity and specificity is not often reported by manufacturers of diagnostic tests. How are they recorded?
independent studies report values in certain populations —> further determined by carrying out specially designed studies from OIE guidelines
How is the definition of diseased and non-diseased determined in diagnostic tests with continuous scales?
determination of cut-off values to assign +ve and -ve status
(continuous scales - ELISA optical density, glucose, ALT, ALP, creatinine, BUN, cell counts)
When do tests on a continuous scale develop false positives?
cutoff miscalculates healthy animals as diseased
When do tests on a continuous scale develop false negatives?
cutoff miscalculates diseased animals as healthy
When are tests on a continuous scale considered a gold standard?
cutoff does not miscalculate diseased and healthy anmals
What is the sensitivity and specificity of this test like?
not very accurate with a lot of overlap for false positives and negatives - may not be worth time or money
What is the classic presentation of diagnostic test values? How is sensitivity and specificity calculated?
2x2 tables
a = true pos
b = false pos
c = false neg
d = true neg
How is accuracy calculated based on 2x2 tables?
diagonal
What is apparent prevalence? How is it used to estimate true prevalence?
estimating disease prevalence with an imperfect test - what seems to be (not actually happening in real life)
use sensitivity and specificity of the test
Lou is a 5-year-old Catahoula Leopard intact male dog from Louisiana. He was tested for Dirofilaria immitis using the IDEXX SNAP test with 85.8% accuracy, 84.1% sensitivity, and 96.9% specificity. If he tests positive, how confident are we that Lou is truly infected and needs to be treated? If he tests negative, how confident are we that Lou does not need to be treated?
- prevalence of heartworm in Louisiana is 5.78%
PPV = p(D+|T+) = a/(a+b) = 49/79 = 62%
- given that Lou was positive, there is a 62% probability that he has at least one female heartworm
NPV = p(D-|T-) = d/(c+d) = 912/921 = 99%
- given that Lou was negative, there is a 99% probability that he truly doesn’t have heartworms
(makes sense - low prevalence = high NPV and low PPV)
What is the sensitivity, specificity, and accuracy of the test in this 2x2 table?
What is an important note about conditional probabilities?
p(A|B) =/= p(B|A)
What do predictive values depend on?
PREVALENCE - study values CANNOT be used for patients (studies in other countries or states are not the same in Louisiana regarding Lou)
How do predictive values differ with low prevalence and high prevalence?
LOW PREVALENCE:
- PPV decreases: less accurate
- NPV increases: rare disease upon negative result, almost 100% sure animal is not infected
HIGH PREVALENCE:
- PPV increases: good accuracy of positive result
- NPV decreases
A practice is using an FeLV test with a sensitivity of 90% and a specificity of 95%. Assuming the prevalence of feline leukemia in the area is 5%, what is the negative predictive value of the test?
140 wallabies are serologically tested for disease. 35 test seropositive and 105 test seronegative. However, postmortem data reveals 5/35 of the seropositive wallabies are disease-free and 4/105 of the seronegative are diseased. What is the positive predictive value of the serological test?
Which of the following tests has the highest specificity for 200 cats, 100 with disease and 100 without?
a. 50 true positive, 80 true negative, 50 false negative, 20 false positive
b. 95 true positive, 65 true negative, 5 false negative, 35 false positive
c. 85 true positive, 85 true negative, 15 false negative, 15 false positive
d. 75 true positive, 95 true negative, 25 false negative, 5 false positive
specificity = test negative given that the animal is diseasd —> truly negative!
D
What are screening tests applied to? What are the 2 characteristics?
apparently healthy members of a population to detect the presence of disease —> positives are subject to further diagnostic workup
- cheap, easy
- high sensitivity, where negative individuals are considered truly non-diseased (SnOUT)
What are confirmatory/diagnostic tests applied to? What are 3 characteristics?
confirm or classify disease status in sick patients with clinical signs —> all animals are abnormal
- more technical equipment
- applied to small number of animals
- high specificity, where positive individuals are considered truly diseased (SpIN)
What are 5 possible reasons for lack of sensitivity?
- antibodies not produced or in low levels
- antibodies present, but blocked by inhibitors
- lab error - kit production/use
- unrepresentative samples of body
- company cut-off setting is too high
Why are tests with high sensitivity used for screening?
when disease is hard to find (low prevalence), you don’t want to miss sick ones (false negatives) —> highly sensitive tests have low false negatives, so negative results are truly negative and can be ruled out
(SnOUT)
What are 4 reasons for lack of specificity?
- artificially induced immunity (vaccine generates antibodies that cross-react with test)
- contamination
- cross-reaction
- company cut-off setting too low
Why are tests with high specificity used for confirmation?
used as a confirmation of a previous test; don’t want healthy animals to be called positive —> highly specific tests have low numbers of false positives, so positives are truly positive and can be ruled in
(SpIN)
What type of test is useful when disease is rare (<1%)?
sensitive
- specificity is rarely high enough to give adequate PPV
How can tests with different levels of sensitivity and specificity be used for?
When can an individual be declared positive in parallel tests? How does this affect sensitivity and specificity?
if at least one of the multiple tests returns positive
- INCREASES sensitivity —> increases NPV
- DECREASES specificity —> decreases PPV
When can an individual be declared positive in series tests? How does this affect sensitivity and specificity?
if ALL tests return a positive result
- INCREASES specificity —> increases PPV
- DECREASES sensitivity —> decreases NPV
(more likely that animals will be missed)
What is herd-level testing (pooling)? What 4 parameters are included?
test on more than 2 animals in a herd (not whole herd), mixing together several samples to generate a single sample that will be tested
- individual-level Se and Sp of the test
- prevalence within herd (P)
- number tested in the herd (n)
- number of reactors per group to designate positive herd (1 or more positive = positive herd)
What are the 3 general trends in herd-level testing?
- HSe increases as prevalence within herd increases
- HSe increases as number tested in the herd increases
- HSp decreases as Sp decreases
What are the 2 possible reasons for a diseased detection outcome in a herd-level test?
- diseased animals reacted to the test —> reflects test’s sensitivity
- non-diseased animals reacted —> reflects test’s lack of specificity
false positive reactors could correctly specify that the herd is infected even though no actually diseased animals tested positive
What do likelihood ratios reflect? What do the values of LR mean?
direction and strength of evidence provided by a test result
LR < 1 = supports ABSENCE of condition
LR > 1 = supports PRESENCE of condition
further away from 1 = stronger evidence
(highly sensitive or specific tests will have strong likelihood ratios)
What are the 3 components of Fagan’s nomogram?
- prior probability - prevalence before the test
- likelihood ratio
- posterior probability - prevalence after test
What are the 5 components of a two-step nomogram?
- prior probability - prevalence before test
- diagnostic sensitivity
- likelihood ratio
- diagnostic specificity
- posterior probability - prevalence after test
What are clinical decision thresholds?
probability (confidence) of your patient having a given condition based on the evidence gathered and tests done
