11b – Clinical Diagnosis II Flashcards
What happens as the prevalence drops in terms of PPV and NPV?
- NPV increases to very HIGH levels
- PPV falls dramatically
What does being a member of a high risk group (high prevalence group) due to PPV of a test for that disease?
- Increases PPV
What are the best tests to RULE-OUT disease?
- Negative test with high sensitivity and NPV
What are the best tests to CONFIRM (Rule-in) disease?
- Positive test with high specificity and PPV
What can you change to get a better result?
- Can NOT change the risk group of the patient
- Can use other tools
At what values of prevalence of a disease do diagnostic tests provide the most insight?
- 40-60%
- INTERMEDIATE
What happens when prevalence decreases?
- Increased NPV
- Drops after the 40% point and then drops quickly
What happens when prevalence increases?
- Increased PPV
- Around/between 80-90: not much changes
When do diagnostic tests perform the best?
- When used in the ‘toss-up’ scenario
- *pretest probability of disease is near 50% and predictive values are maximized!
What happens if pre-test probability is very high?
- Do NOT gain much info by testing
What happens if pre-test probability is very low?
- Probably can’t justify even doing the testing
How do we optimize predictive value?
- Use in situation where pre-test probability of disease is around 50%
- After using one test, apply another more specific test to positive animals
- *use 2 test concurrently
What is parallel testing?
- 2+ different tests are performed and interpreted simultaneously
- *Animal considered positive if reacts positively to one OR both tests
- Increased sensitivity and NPV
- *more confident in negative test results
- Patient must prove it is healthy!
What is serial testing?
- Test conducted sequentially based on results of previous test
- Usually use one test, then apply a more specific test to those that test positive
- *only classified as positive IF it is positive on BOTH tests
- Maximizes specificity and improves PPV
- *patient must prove it has the condition
What is repeat testing? Negative (herd) re-testing
- Modification of serial testing
- *intermittent testing
- Test negative animals are re-tested with the same test at REGULAR INTERVALS
- Forms the basis of test and removal programs designed to eradicate disease
- Improves aggregate-level sensitivity
- Ex. Johne’s disease, Trichomoniasis, Salmonella
Example: How many preputial scrapings and cultures should we do to be fairly confident that the bully is truly negative for trichomoniasis?
- Figure out how confident you are if the test result comes up NEGATIVE
- *immunity does not actually last that long
- *need to test 3 times!
o Don’t shed consistently
Young, yearling bull (5% pre-test probability)
- 1st and 2nd test: 99% NPV
Bull (40% pre-test probability)
- Takes 3 tests to get 99% NPV
- 1st test: 90% (10% chance of losing calf crop)
- 2nd test is only taking the negative bulls
o 98% NPV
Older, mature bull (90% pre-test probability)
- Don’t get 99% NPV until 4 tests
- 1st: 41%
- *likely wouldn’t test if the pre-test probability was that high!
Parallel testing example (ex. Johne’s disease)
- Have multiple tests that could give good answers
- *increase sensitivity in final stages of eradication program
- Ex. test herd with ELISA and culture
2 signs for heat detection in dairy cows
- Mounted
- Vulvar
If you are trying to rule out a disease, what do you want your sensitivity or specificity to be? (EXAM)
- High sensitivity
- High negative predictive value
- *works best when pre-test probability of disease is LOW
- SnNOUT
If you are trying to rule in a disease, what do you want your sensitivity or specificity to be? (EXAM)
- High specificity
- High PPV
- *works best when pre-test probability of disease is HIGH
- SpPIN
What is the cost of a FALSE NEGATIVE test? (ex. exotic disease (FMD)=disastrous consequences) (EXAM)
- Highly sensitive tests, even at the cost of specificity
- Avoid FALSE NEGATIVES at all costs
- *multiple tests should be interpreted in PARALLEL
What is the cost of a FALSE POSTIVE test? (EXAM)
- High treatment costs
- Treatment that are potentially dangerous
- Euthanasia of valuable animal might be possible
- *use highly SPECIFIC tests
- *multiple tests should be interpreted in SERIES
What happens when you do parallel or negative re-testing?
- False negatives are DECREASED
- Sensitivity increases
- NPV increases
What happens when you do serial testing?
- False positives are DECREASED
- Specificity must increase
- PPV increases
Choosing a cut-point
- Continuous outcome for a diagnostic test (Ex. enzyme or Ig levels)
- Distribution of test results OVERLAP
- *Impossible to find a cut off point which PERFECTLY discriminates between 2 groups
- FLEXIBLE
- Intermediate zone: if in this zone=re-tested after a certain time period
What happens if you select a low cut off value?
- Get good SENSITIVITY
- False negatives are NOT acceptable
- False positives can be controlled by a 2nd more specific test on initial positive results
o Consequences of false positives are NOT severe - Disease can be treated
o UNTREATED cases are FATAL
What happens if you select a high cut off value?
- Get a good SPECIFICITY
- False positive results are NOT acceptable
- False negatives results are controlled by using a second test in parallel
- False negatives consequences are NOT severe
- Disease is SEVERE, but confirmation has little impact on therapy and prevention
Lead poisoning
- Accumulates in bone
- Transferred across placental barrier
- Excreted in urine, bile, feces and milk
- *can not put lead in food chain
o Some have high lead blood levels, BUT no clinical signs
What can you use to set cut points?
- Receiver operator characteristic (ROC) curve
What is a receiver operator characteristic (ROC) curve?
- True-positive rate on vertical (sensitivity)
- False-positive rate on horizonal (1-specificity)
- *point closet to top LEFT CORNER will maximize sensitivity and specificity
o Should consider costs of false positives and false negatives when selecting a cut-off point
o WANT THE LARGEST AREA UNDER THE CURVE
ROC curve with a diagonal line
- 50/50 coin flip!
What is mass screening?
- Sampling volunteers or a sample of the population to detect disease
- Ex. Brucellosis testing
- Ex. Bovine TB testing
What is case finding?
- Seeking an EARLY diagnosis when a client brings animal to vet for unrelated reasons
o Routine physical exams
o Routine fecal exams
o Feline leukemia virus
o Heartworm testing
o CMT testing of quarter milk
o Meat inspection
o EIA serologic testing
At what point are you trying to intervene?
- Close to early diagnosis being possible
- *before animal gets sick and clinical diagnosis!
What tests should be used for screening tests?
- Hard to estimate sensitivity
- SPECIFICITY is most important
- PPV is only measuring diagnostic test performance, not EFFICACY of screening
How do you tell if a disease can be detected before routine diagnosis and it’s early detection is worth our effort?
- Patient lives longer
- Quality of life improves
- Treatment costs decreases
- *early diagnosis will almost always IMPROVE survival even if the therapy applied is useless
How can you evaluate screening programs?
- Randomized clinical trials
- Randomize patients to receive screening test or not
- *caution in interpretation of outcomes
What are some types of biases? (EXAM)
- Volunteer effect
- Zero time shift or lead time bias
- Length time bias
What is the volunteer effect?
- Clients that bring animals for screening tests are NOT the same as those that don’t
o Better management
o Improved health status - *more likely to LIVE LONGER
What is zero time shift or lead time bias?
- Comparing survival times after early diagnosis to survival times after conventional diagnosis
What is the ‘zero point’ for survival time?
- Time of diagnosis
What is lead time bias?
- If early diagnosis takes place before conventional diagnosis
- *lead time is NOT taken into account
o May be creating an extra year ‘backwards’ of disease NOT a year more of survival
What is length time bias?
- Long pre-clinical phase diseases usually have a long clinical phase
- Short pre-clinical phase disease usually have a short clinical phase
- *disease that are more likely to be detected by screening tests will SURVIVE LONGER THAN THOSE THAT ARE NOT
o Going to find those with the ‘less severe’ disease
o Miss those severe ones as they aren’t around as long!
What are the hazards of early diagnosis?
- Need to be sure of efficacy when marketing our treatment to clients
- False positive risk: especially important if there is a debilitating treatment involved
- Labeling
Diagnostic panels
- Each test has it’s own sensitivity and specificity
- *probability of a false positive occurring on at least one of two tests can be calculated!
What are the objectives of herd testing?
- Determine prevalence of infected herds
- Certify herds as disease negative for eradication or trade
- Examine risk factors for herd level disease
What are the differences between herd and individual testing?
- Uncertainty around individual Se and Sp is AMPLIFED in HSe and HSp
- Bias in AMPLIFIED
- *impact of false results if generally GREATER
A positive test on a herd basis is not a
- Positive diagnosis
- *false positives can occur in clean herds
- Disease prevalence drops in the herd, PPV of screening PROGRESSIVELY gets worse
- *need tests with VERY HIGH specificity when prevalence is low
What happens to HSe and HSp when you increase the number tested?
- HSe increases
- HSp decreases
What happens to HPPV and HNPV when you increase the number tested?
- HPPV decreases
- HNPV increases
What happens to HSe and HSp when you increase the required reactors to be considered positive? (‘number needed to determine the herd is positive)
- HSe decreases
- HSp increases
What happens to HPPV and HNPV when you increase the required reactors to be considered positive?
- HPPV increases
- HNVP decreases
When are pooled samples ideal?
- When within-herd prevalence is LOW
What are the pros of pooled samples?
- Decreased laboratory cost
- Increased HSe due to increase n (sample number)
What are the cons of pooled samples?
- Risk of decreased Se due to dilution
- Logistical challenges of mixing sample
- If one sample is contaminated (example blood + feces are inhibitors of PCR) than it will affect all of them
What can you do to evaluate a test if there is not gold standard?
- Comparing 2 tests
- Comparing agreement between 2 clinicians
- Comparing agreement within clinicians (will they give the same diagnosis if given same data more than once?)
What is the kappa statistic?
- PROPORTION of agreement measured BEYOND THAT EXPECTED BY CHANCE ALONE
- Don’t need to know the formula
What is another option if there is no gold standard?
- Index test is compared to a reference standard
o Se and Sp obtained relative to reference test
What are 2 latent class analysis?
- Maximum likelihood estimation (MLE)
- Bayesian
Maximum likelihood estimation (MLE)
- Independence between tests required
- Observed data
Bayesian
- Allows for correlated tests
- Prior information and data
- Ex. serology for ‘historical’ exposure and culture for current exposure status
