Test Validation Flashcards
What is the purpose of validation?
To establish the truthfullness.
What is a test compared to?
a “gold standard” for accuracy.
Gold standards used for VU comparison are typically what?
- Angiography
- CTA
- MRA
- Pathology
- Re-test same day/ different tech (DVT)
What are the 4 steps in a test validation process?
- Question defined.
- Data log created and completed.
- Calculate the statistics.
- Report te statistics.
How should you define the question?
- Presence of disease.
- Severity of disease.
- Location of disease as defined.
How should a data log be designed?
- Set time limit for data
- Set population description.
- ie. patients with no intervention in carotid.
- Set parameters to be collected.
- log according to diagnostic criteria grade as indicated in the conclusion.
What are some tips for collecting data?
- Stay blinded to the opposite test while completing log.
- Keep a list of ALL exams thrown out.
When is a matrix used?
It is used when there are more than 2 categories for test results.
What are 3 parts of a matrix statistics?
- precent agreement
- percent overestimated
- percent underestimated.
What is the percent agreement?
Diagonal boxes on diagram.
How do you find the percent agreement?
(# matched)/(total)
What is the percent overestimated?
Duplex hugher grade than angio.
How do you find the percent overestimated?
(# overestimated)/(total)
*above diagonal
What is the percent underestimated?
Duplex lower grade than angio.
How do you find the percent underestimated?
(# underestimated)/(total)
*above diagonal.
When can more specific statistics be used?
If only two answers are possible to answer the question,
What positive/negative statistic is used to answer this question:
What percentage of patients who come to the lab with disease do you identify as having disease?
Sensitivity (SENS)
What positive/negative statistic is used to answer this question:
What percentage of normal patients can your testing identify as normal?
Specificity (SPEC)
What positive/negative statistic is used to answer this question:
Of all the patients with positive test results (with disease), what percentage actually have disease?
Positive Predicitve Value (PPV)
What positive/negative statistic is used to answer this question:
Of all the negative test results (normal) that your lab reports what percentage of those are actually normal?
Negative Predictive Value (NPV)
What positive/negative statistic is used to answer this question:
What is the overall accuracy of your lab results?
Overall Accuracy (OA)
If both gold standard angio and duplex match what does it indicate?
True positive.
If both standards dont match what does it indicate?
True negative.
If the gold standard for angio is negative, and for duplex is positive, then what does it indicate?
False positive.
If the gold standard for angio is positive and for duplex is negative, what does it indicate?
False negative.
How do you calculate the sensitivity?
TP/ALL positive angios
How do you calculate specificity?
TN/ALL negative angios
How do you calculate PPV?
TP/ALL positive duplex
How do you calculate NPV?
TN/ALL negative duplex.
How do you calculate the OA?
TP+TN/Total Comparison
What number does the Overall Accuracy (OA) yield?
A number between the sensitivity and specificty and between the PPV and NPV
What is sensitivity?
The ability to idenitfy disease in all patients who are tested with the disease.
What is specificity?
The ability to identify patients without the disease in all patients tested without the disease.
What is PPV?
The probability that a single positive duplex result is truly positive by the gold standard.
What is NPV?
The probability that a single negative duplex result is truly negative by the gold standard.
