Appraising & Applying Research on Diagnostic Tests Flashcards
Why learn about this topic?
- Expanding scope of Rx practice & availability of point of care tests (POCT)
- Why Lab Testing?
– Detect Disease, Guide treatment, Monitor treatment response, Monitor disease progression - Push for pharmacy-based testing/screening services (Creation & Application of Standards of Practice)
– Testing for minor ailments, Strep, H. Pylori, Influenza, (?COVID), HIV
– Identification of asymptomatic individuals with Diabetes, HTN, dyslipidemia, osteoporosis,
CKD, Hep C, etc…
– Risk screening using clinical prediction rules (e.g., CANRISK, CV Risk)
– Testing to monitor patients with established conditions. - Cautions
– Respect Rx scope of practice…
– Context of overdiagnosis & unnecessary testing
Management of Pharyngitis
Strep Throat POCT - What you’ve already learned (PHARM 346 & Skills Lab)
- Cannot diagnose strep throat with symptoms alone, & cannot diagnose with lab
- Cannot diagnose strep throat with symptoms alone, & cannot diagnose with lab
test alone. - Centor Score to help determine who needs a throat swab.
- Positive tests may indicate carrier status, not active infection.
- Gold standard: Throat swab for culture.
- POC Rapid antigen test
– If Rapid antigen test positive – follow-up throat culture not required.
– If Rapid antigen test negative – generally recommended to confirm with throat culture for children/adolescents, NOT adults.
We need to have a symptomatic patients. Because when we do lab tests, we are, we can just detect patients who just have colonization with strap in their throat. And if they’re not symptomatic, then they don’t have an indication for treatment.
Centro score: not a really good test to determine whether or not a person has strep throat. It actually helps us to determine whether or not we should do another test.
- Canadian Pediatric Society: “Do not routinely do a throat swab where children
present with a sore throat if they have cough rhinitis or hoarseness as they almost
certainly have viral pharyngitis.”
Main Issue: Pharmacists may miss kids
with GABHS and they
Main Issue: Pharmacists sending
patients with Positive tests but no
symptoms to emerg….
How accurate are POCT in diagnosis of
GABHS Pharyngitis?
Terminology
* Sensitivity, Specificity?
* Ruling in/Ruling out disease?
* Positive Likelihood ratio?
* Negative Likelihood ratio?
HELP!!! What does all of this mean? How do
interpret & apply this info?
What is an evidence-based approach to diagnosis?
How do we evaluate a study of a diagnostic test?
point-of-care testing, including rapid antigen detecting, detection tests and newer tests that are based on nucleic acid detection are useful for ruling in a diagnosis of group a beta hemolytic strep when positive. Because they have high specificity, 95 to 99 per cent. The nucleic acid tests may be more sensitive than the rapid antigen detection test.
with immediate testing, treatment may not always be required.
Pattern recognition vs Probabilistic
diagnostic reasoning
Pattern recognition vs Probabilistic diagnostic reasoning
See it and recognize disorder
vs. Clinical assessment generates pretest probability
Compare posttest probability with thresholds
vs. New information generates posttest probability
(usually pattern recognition implies
probability near 100% and so above
threshold)
vs. (May be iterative)
Compare posttest probability with thresholds
Pattern rec: One of the most classic examples of using pattern recognition to make a diagnosis is in the area of Dermatology
- Fast way of doing diagnosis
Probabilistic
- Slow way
- when someone is admitted with potential heart failure, that there are other potential explanations for the patient’s symptoms, shortness of breath, etc. So it could be heart failure or it could be pneumonia,
They set up a few different differential diagnosis that are the ones that are the most probable. - they estimate what the probability of the disease is using the information that they're collecting as part of their clinical assessment. So gathering history, doing a physical exam, you can gather information and understand more by knowing what the patient is presenting with, whether they were febrile, how short of breath is, what their other medical history is, what their diseases are - clinical assessment generates a pre-test probability once you have collected a point-of-care test, that new information gathered as part of that test generates what is called a post-test probability.
Test and Treatment Thresholds
Probabilistic diagnostic reasoning
Pretest probability: Probability of target condition being present before results of diagnostic test are known.
Posttest probability: Probability of target condition being present after results of diagnostic test are known
The goal is to reach a conclusion that is strong enough to Act on…
– The relationship between the posttest probability and threshold probability determines the action.
probability below test threshold: no testing warranted
probability between test and treatment threshold; further testing required
probability above treatement threshold; testing completed; treatment commences
somewhere close to 100%, we would say that the person has strep throat, that we’ve exceeded the treatment threshold, we are confident enough that the patient actually has strep throat. no furhter testing
We are thinking that it is not strep throat. What the course of action in that situation, if it was close to zero, when the probability is below a test threshold, we would reassure the patient and say, I don t think that you have strep throat. We don’t need to do any further testing because I’m confident that you don’t have strep throat.
many situations is actually that the probability of the disease, the patient having the disease, is usually going to fall somewhere between the test and the treatment threshold.
This is where you need to do more testing in order to either become more confident that the person has the disease by increasing the probability of the disease
we want to get enough information so that we can exceed the treatment threshold and decide that we’re going to treat the patient or we’re going to be below the test threshold. Know that there’s a low probability of disease where we can reassure the patient
Let’s Think about
Diagnostic Test Studies
The JAMA Users’ Guides Approach
Lets Start with —> What are the Results?
* What Likelihood ratios were associated with the
range of possible test results?
– Let’s Review Test performance (Sensitivity, Specificity)
– Introduce Likelihood Ratio
How Serious is the Risk of Bias?
* Did participating patients present a diagnostic dilemma?
* Did investigators compare the test to an appropriate,
independent reference standard?
* Were those interpreting the test and reference standard
blind to the other results?
* Did investigators perform the same reference standard to
all patients regardless of the results of the test under
investigation?
Lets Start with —> What are the Results?
* What Likelihood ratios were associated with the
range of possible test results?
– Let’s Review Test performance (Sensitivity, Specificity)
– Introduce Likelihood Ratio
What are the Results?
* What Likelihood ratios were associated with the range of
possible test results?
How can I apply the results to patient care?
* Will the reproducibility of the test results and the
interpretation be satisfactory in my clinical setting?
* Are the study results applicable to the patients in my
practice?
* Will the test results change my management strategy?
* Will the patients be better off as a result of the test?
Sensitivity & Specificity
Exam Tip…. Setting Up Your 2 x 2 Table for Dichotomous Test Results
set up the dichotomous table
defn of sensitivity and specificity
Sensitivity= Likelihood of a positive test
when disease is present
Specificity = Likelihood of a negative test
when disease is absent
Performance of the Strep Rapid Antigen Detection
Test vs. Reference Culture
see slide 18
calculate sensitivity, specificty, prevalence, accuracy
Sensitivity
Clinical Interpretation
Definition
* Proportion of individuals with disease that have a
positive test result.
– (Aka Truth in those with Disease)
- If a test has a low sensitivity…
– High possibility of a false negative
– And is is negative, need to do more testing… if you don’t do more testing (i.e., rely only
on this test) a person with the condition will not be treated (undertreatment). - Sensitivity helps us rule out conditions, it does NOT give us confidence that someone with a positive test has the condition…
- If test has a high sensitivity…
– Low possibility of a false negative
– And is negative, the result tends to rule out the condition (SnNout)
Specificity
Clinical Interpretation
Proportion of individuals without disease that
have a negative test result.
– (Aka Truth in No disease)
- If a test has a low specificity…
– High possibility of a false positive
– And is is positive, need to do more testing… if you don’t do more testing (i.e., rely only
on this test) a person without the condition will be treated (overtreatment). - Specificity helps us rule IN conditions, it does NOT give us confidence that someone with a negative test does not have the condition…
- If test has a high specificity…
– Low possibility of a false positive
– And is positive, the result tends to rule in the condition (SpPin)
Limitations of the SpPin & SnNout Approach
SpPin & SnNout Approach Is a Source of Diagnostic error
– The power of a test to “Rule Out” a diagnosis does not exclusively rely
on sensitivity. It also depends on specificity…
– The power of a test to “Rule In” a diagnosis does not exclusively rely on
specificity. It also depends on sensitivity…
Likelihood ratio
The Solution to Understanding Test Performance
- Combines sensitivity and specificity into a single number
- “Likelihood ratio expresses the magnitude by which the probability of a diagnosis in a given patient is modified by the result of a test.”
- They are dimensionless numbers
- LR may be used with:
– Symptoms (i.e., history)
– Physical exam
– Lab tests
– Imaging procedures
– Scoring systems (i.e., Clinical Prediction Rules)
Likelihood Ratios
Positive Likelihood Ratio (LR+) Negative Likelihood Ratio (LR-)
Total TP + FN FP + TN
Likelihood of a negative test in the presence of disease
as compared to the likelihood of a negative test in the
absence of disease
= FN Rate / TN rate
= (FN/(TP+FN)) / (TN/(FP+TN))
Likelihood of a positive test in the presence of disease
as compared to the likelihood of a positive test in the
absence of disease
= TP Rate / FP Rate
= (TP/(TP+FN)) / (FP/(FP+TN))
= Sensitivity / (1-Specificity) ** = (1-sensitivity) / specificity **
Likelihood ratio
Clinical Interpretation
A positive LR (LR+) is the
strength of a positive test result
on a scale of 1-100
- A negative LR (LR-) is the
strength of a negative test
result, where lower is better on
a scale of 1 to 0.01.
Likelihood Ratios
Performance of the POC Tests vs. Reference Culture in Plain English
Rapid Antigen PCR Nucleic Acid
Sensitivity = 84.5% Specificity = 95.3%
LR(+) = 18 LR(-) = 0.162
Sensitivity = 97.7% Specificity = 93.2%
LR(+) = 15 LR(-) = 0.025
LR+ A Positive RADT test result is 18 times
more likely in patients with Disease than
without Disease.
LR- A Negative test is about a tenth as likely
in patients with disease than without disease.
LR+ A Positive rapid Nucleic acid test result is
15 times more likely in patients with Disease than
without Disease.
LR- A Negative rapid nucleic acid test is about a
hundredth as likely in patients with disease than
without disease.
Likelihood Ratios
Using LR to Guide Clinical Decision Making in Individual Patients in Practice
Pocket Card
online Calculator
App-based Calculator
Using the Fagan Nomogram
* Pretest probability is the anchor
* Adjust the Pretest probability using the LR from the test as a multiplier
* Have to use a nomogram or online calculator because
* Pretest probability is depicted as probability (i.e., risk) (%)
* While LR is depicted as odds
Back to our Case
“Hi my child has a sore throat. What do you recommend?”
Pretest probability
* For GABHS pharyngitis is reported to peak b/w 5 to 10
years old.
* The prevalence is reported to be:
– Children: ~20-25%
– Adults: ~5-10%
* Use of Centor Score. Her modified Centor score was 5.
– For simplicity, lets just use the non-modified Centor and assume her
score was 4.
Likelihood ratio
* LR for strep throat POCT
– Let’s use the numbers from the TFP summary based on SR rather than the one individual study (i.e., Wang)
– Rapid Antigen: LR+ 16.8; LR- 0.16
– Nucleic acid detection: LR+ 92; LR- 0.08
Post Test Probability: ?
Pulling it all together
Back to Our Case; Lets Practice “Warming Up” or “Cooling Off” to a Diagnosis….
Note:
* Experienced clinicians estimate pretest and post
test probabilities intuitively using the anchoring
and adjusting rule of thumb.
* LR can help to calibrate clinical intuition and
how much your thinking should shift based on a
positive or negative test result
* Tell you how much you should warm up or cool
off to a diagnosis based on the strength of a test
result.
Pre-test probability: Assuming of 56%
Test Result: RADT is positive: (LR + 16.8)
Post test probability = 96%
Interpretation: Treatment threshold exceeded
Pre-test probability: Assuming of 56%
Test Result: RADT is negative: (LR + 0.16)
Post test probability = 17%
Interpretation: Further testing required as post test
probability likely NOT below test threshold (clinical
judgment)
LR Rule of Thumb
No nomogram or digital device???
- For pretest probability between 10% & 90% (which is the usual case)
- Need to only remember 3 benchmark LR:
– 2
– 5
– 10 - Multiples of 15%
– LR+ of 2 increase the absolute probability by ~15% (LR- 0.5: decreases by 15%)
– LR+ of 5 increase the absolute probability by ~30% (LR- 0.2: decreases by 30%)
– LR+ of 10 increase the absolute probability by ~45% (LR – 0.1: decreases by 45%)
Some other examples of LR …
In patients presenting with Signs Symptoms of Strep Pharyngitis1
* Fever LR+ = 1.7 LR- = 0.71
* Anterior cervical adenopathy: LR+ = 1.5 LR- = 0.63
* Any exudates LR+ = 2.2 LR- = 0.58
* Absence of Cough LR+ = 1.5 LR- = 0.53
Other Useful Likelihood Ratios JAMA Rational Clinical Exam
* Upper GI Bleed
– Melenic stool LR+ = 25 LR- = 0.52
* Myocardial Infarction
– Pain radiating to the right arm LR+ = 3.8 LR- = 0.86
* Health Literacy
– “How confident are you filling out medical forms by yourself?”
* “A little” or “no confidence” LR+ = 5.0
“Multilevel” Likelihood Ratios
Example of LR with each level of the Centor Score
Another Advantage of Using Likelihood Ratios
* LR can be used when tests do not have dichotomous results (i.e., Not LR+ or LR-)
* Allow use of full range of test results without having to collapse or set “cutpoints
see slide 34
