Lecture 3: Diagnostic Accuracy Studies Flashcards
how well a test actaully tests the thing were trying to measure
Validity
For something its really easy like a goniometer - you’re actaully measuring the degrees you’re trying to measure
Other things are a little harder to measure
* Risk of falling
* Thats a construct - an abstarct variabile - hard to define
so you have to think how valid the test is for measuring something abstract
When assing to see if some test is valid we need to compare it to some gold standard
* is there a best way of measuring this construct as it stands right now - that way we can compare our test to that and see if it really measures well or not
EX: ACL tear gold standard = MRI
* We can compare our diagnostic acuracy of our anterior lacman test to the MRI (and that works with any of those other tests)
If no gold stand exists we have to compare it to our reference standard
* Reference standard = the best tool we have (basically just the best thing we have)
When assessing the validity of the test we also want to know who was measured
For the anterior lachman for example, was this validity measured in the general population? Or was it some subset like the sport population or the geriatric population (basically figuring out where they got their validity # from)
* Is going to tell us what populaion this test is appropriate for
* For example the TUG is good at finding fall risk for the elderly population, not the young population - is not valid for younger population
We also want to know the clinical setting
* Was this validity measured in elderly pts in an outpatient setting? Those pts are going to be a lot different than those in a long term care facility
When assesing the validity we need to know the how
were the resaechers blinded? were the researchers blinded?
If we know the results already taht could scew our data
* if we know that pt X already has an MRI confirmed ACL pathology that could scew my data when im taking the anterior lachman (so we want it to be double blinded)
* It could bias your results if you already know
We want to think about who performed the measures
* Was it professionals or just anybody
also need to know the when
* The timing of the test
* Think about doing it in the acute stage (everything painful / positive) vs chronic (everything calmed down a little bit)
Same result over and over
Relability
whether its in the same person (intra) or between people (inter)
Dx = disease
Positive Test with disease = true positive
* a
Positive test without the disease = false positive
* b
* not ideal because it could lead to more testing and treatment that isnt necessisary
* psycological problems
* cost
Negative test with the disease = false negative
* c
* Delays in treatment
* false sense of secruity
Negative without the disease = true negative
* d
Sensitivity and specificity
* indicators of how good a test is
Sensitivity: The porportion of people with the disease that test positive for it
* a/a+c
* A high sensitivity means the porportion of true positives is high and that the porportion of false negatives is low
Specificity
* The porportion of people without the disease who test negative
* A high specificity means the porportion of true negatives is high and the porportion of false positives is low
* d/b+d
This is an example of how to use sensitivity and specificity
What are good screening tests and why?
Highly sensitive tests are good screening tests
This is because the porportion of false negatives is low
SNOUT: High sensitivity test rules out if they test negative for the condition
What test is a good confirmitory test?
* Why
High Specificity tests
because they have a low chance of having false positives (meaning they’re good at ruling in the disease SPIN)
* high specificity rules in
KNOW: A perfect test will have a sensitivity and specificity of 100%
* the closer it is to 100% the better the test is at confirming or excluding the disease
Sensitivity: If the test is highly sensitivie that means that a negative test rules out the diagnosis
* If it came back negative that means that the person likely does not have the disease
* Its a screener
* True positive rate –> out of those who have the diagnosis
* Out of all those who actually have the diagnosis how many actually tested positive
* because if you’re really good at picking up positive results, the 1 time you get a negative we can be sure its a true negative
Specificity: If the test is highly speicific that means that a positive test rules in the diagnosis
* Good at confirming the patient has the disease
* If they test positive they really have the pathology
* This is our true negative rate in those that don’t have the diagnosis
* Out of those who don’t have it, how many actaully test negative
* So if they’re really good at picking up all the negatives, the one time we get that positive result, we can be sure its a true positive result
NOTE: We want both highly sensitie and specific tests
* both are good if they are close to 1 and bad if they are close to 0**
EX: highly specific test. I’m vaccuming and picking up all the little dust particals really well, those are my negatives. Really good at picking up all those little dust particals. Then I roll over something bigger and it crunches in the vaccume. That was a positive. I can be pretty sure that wasnt a little dust partical, it is something else.
EX: Car alarm. Sensitivity is really good at picking up what is not important –> that would be out positive results. This is the wind and a leaf touching the car. But when someone actually bumpbs into the car going of thats something important????
If the test is really good at doing one thing, and something on the other spectrum happens, than were pretty sure thats what it is
Most important to remember SPIN and SNOUT
* Specific test positive rules in
* Sensitivive test negative rules out
The likelihood that a person who test positive actaully has the disease
Positive predicitive value
What does a high positive perdictive value mean?
Provides a strong estimate on who has the disease
The likelihood that a person who tests negative actually does not have the disease
Negative perdictive value
out of all those that tested negative, how many were a true negative
If I did an anterior lachman on someones knee and they tested negative, it would liklihood that she does not have that pathology
What does a negative perdictive value tell you?
Provides a strong estimate of who is disease free
The probability of a diagnosis being present before we perform a diagnostic test
Pretest probability
basically just our initial best guess
The probability of the presence of a disease following a diagnostic test
Post test porabibility
What is a diagnostic test?
* What is sensitivity for this test
* What is specificity for this test?
* If positive perdictive value was really high would that be a good diagnostic test?
* If negative perdictive value was really high would that be a good diagnostic test?
Test thats really good at ruling in some pathology (basically diagnosing it)
Sensitivity = don’t know, this is the tests ability to rule out, and diagnostic tests are ruling some pathology in
Specificity = High (rules in)
Positive Perdictive Value = High
* If they have the pathology and take the test they will test positive - because this is a diagnostic test
* of all the people that tested value a lot are true
Negative perdictive value = we don’t know
* this is a diagnostic test
KNOW: Know that Specificity and PPV are high in diagnostic tests
* Diagnostic test = test that has the ability to rule in
In a good screening test which variables are high
* Spicificity
* Sensitivity
* Positive Perdictive Value
* Negative perdictive value
Sensitivity - these tests are really good at ruling out if you test negative
Negative perdictive value - this is the likelihood that a person who tests negative actaully does not have the disease - we want this high to know our results are true
* This will provide us a strong estimate of who is disease free
So our patient has signs and symptoms of adhesive capsulitits
* so my pretest proabibility for shoulder instability is really low so I wouldnt even do that test item cluster (look at flow chart below) - so i can consider a different diagnosis
* If my pretest proability of that disease actaully being present is really high than I wouldnt even want to do diagnostic tests (whats the point) - however, the threshold on when to do these tests is different for everyone, need to figure out this threshold on your own - also insurance / time play a role - sometimes you don’t even need to do a diagnostic test (think if they came in w/ an MRI confirmed RTC tear, and all the things that they’re saying match up with this - well I wouldnt need to perform diagnostic testing)
However, if I think its adhesive but im not super sure (its in the intermediate condifdence range) than I would test and see where I am now (what my confidence is at my post test proabibility)
* If its high –> the test confirmed it, then I go ahead and treat
* If im still not sure after one test than I would need to do another test / finish cluster
* However, if the test is negative, then I need to switch to a different diagnosis
EBP as it releates to differential diagnosis
KNOW: Likelihood ratio is the ratio over sentivitity of 1-specficity
The lower the LR ratio = the better it is at screening and ruling
The high the LR ratio = the better for ruling in
LR = How many times more likely a positive result will occur in those with the diagnosis compared to those w/o the diagnosis
* EX: For a test with a LR of 5
* A positive test is 5 times more likely to occur in someone who has that diagnosis
KNOW: .5-2 is unimportant
* falls there it doesnt tell us if it occurs more or less
* if its outside of that range its important
* so if the test falls between 0.5-2 its unimportant
Positive LR: Tells you how much more likely a positive test result is to occur in someone with the condition compared to someone w/o it
* The higher the LR the better the test is at identifying those with the disease
* the higher the LR the better the test is at identifying those w/ the disease
Negative LR: Tells you how much less likely a negative test result is to occur in someone with the condition compared to someone w/o it
* the lower the LR the better the test is at ruling out the disease
LR+ >1: The test result is more liekly to be positive in patients with the condition
LR- <1: The test result is more likely to be nehative in patients w/o the condition
How many times more likely a positive result will occur (positive test) in those with the diagnosis compared to those w/o diagnosis
Positive likelihood ratio
Stronger test will have a higher LR
LR+ = 8.8: A positive result is 8.8 times more likely to occur in those w/ the diagnosis vs those that don’t have it
The likelihood ratio because just tells us how likely it is that a patient has the dx following the test
What is the un important range?
.5-2
How many times a negative result will occur in those with the dx vs those who do not
Negative likelihood ratio
A strong test will have a low LR-
Lets say our pre test proabibility of 20%
So if I get a negative result with a negative LR of 0.05, that takes me from 20% sure to 1% sure they have the diagnosis
* basically confirms that I wasnt confident and now Im really not confident that that is what it is
Should pt 90% sure she has adhesive capsulitits
* Do test for capsular pattern and its negative and that has a negative LR of 0.05
* That negative result brings me down to a post test porabibility of 30%
So I think every different kind of test has both a positive likelihood ratio (if the test result is positive it runs it through this) and a negative likelihood ratio (if the test is negative it runs it through this)
LR < 1 = negative = decreases post test proabibility of this actaully being the problem
LR > 1 = positive = increases post test probaibility of this actaully being my problem
She said you want ~70% post-test proabibility to move on w/ diagnosis
TEST: will be given something like the chart below and given 3 points
* EX: given a point at pre test a point on the LR and a point on post test proabibility and ask what it means
As long as the sensitivity and specificity are found for a specific test we can find the negative and the positive LR on any test (which is what we would use to put through that graph)
So if I had a negative test I would look at its negative LR
KNOW: Test item clusters can have their own LR
* essentailly that LR can vary depening on how many are positive
* If 3/4 are positive than +LR = 3
* If 4/4 are positive than +LR = 10
* If 0/4 are positive than -LR = .001
notice that at some point that LR flips to negative if none of those tests are coming back positive
This is a guide for assessing diagnostic articales
* which diagnostic measures are most useful
* Will tell you the quality of the study
A set of validated variables utilized to predict a diagnosis, clincal outcome, or needed for referral
* say if something is likely going to happen
Clinical Prediction “Rules”
EX: Ottowa ankle rules –> can help us rule out the diagnosis
EX: Wells criteria (if they meet them) –> referral to physician to rule out DVT
Cervical manipulation –> sucess of cervical thrust manipulation
Assists with effective clinical decision making
Ruling something in (to go to another person), ruling something out, or sucess or failure of an intervention
* these are the basic principles to why we have clinical perdiction rules (not this is not a CPG)
