Accuracy Metrics in Epidemiologic Measurements Flashcards
What is validity?
An expression of the degree to which a specified test/exam/measurement measures what it purports to measure; the ability of a test to distinguish between who has a disease and who does not
EX: How often does a screening test for breast cancer label those with breast cancer as having breast cancer? (How good is the test in identifying populations of people with and without the disease in question?)
What are the two components of validity?
1) sensitivity - the ability of the test to identify correctly those who have the disease
2) specificity - the ability of the test to identify correctly those who do not have the disease
How do you calculate sensitivity and specificity?
You must have another source of truth with which to compare the test results. (to find who “really” has the disease and who “does not” from a source other than the test you’re using)
What is the use of sequential (two-stage) testing)?
In sequential (or two-stage) screening, a less expensive, less invasive, or less uncomfortable test is generally performed first, and those who screen positive are recalled for further testing with a more expensive, more invasive, or more uncomfortable test, which may have greater sensitivity and specificity. It is hoped that bringing back for further testing only those who screen positive will reduce the problem of false positives. Then you can calculate net sensitivity and net specificity
How do we calculate the predictive value of a test?
with the positive predictive value (PPV).
PPV = (true positives)/(true positives and false positives
How do you calculate the negative predictive value of a test?
“If the test result is negative, what is the probability that this patient does not have the disease?”
NPV = (true negatives)/(true negatives + false negatives)
Explain the relationship between predictive value (PPV) and disease prevalence.
The higher the prevalence, the higher the predictive value. Therefore, a screening program is most productive and more cost-effective if it is directed to a high-risk target population. Also, the relationship between predictive value and disease prevalence also shows that the results of any test must be interpreted in the context of the prevalence of the disease in the population from which the subject originates.
Explain the relationship between predictive value (PPV) and the specificity of a test.
As specificity increases, PPV increases.
What are the factors (3) that contribute to the variation between test results during repeatability (shows reliability) of tests?
1) intrasubject variation: variation within individual subjects
2) intraobserver variation: variation in the reading of test results by the same reader
3) interobserver variation: variation between those reading the test results
Explain what is meant by intrasubject variation.
Values obtained in measuring human characteristics often vary over time even during 24hrs like blood pressure. This, as well as the conditions under which certain tests are conducted (like after eating, at home or doctor’s office, etc.)
Explain what is meant by intraobserver variation.
Tests and examinations differ in the degree to which subjective factors enter into the observer’s conclusions, and the greater the subjective element in the reading, the greater the intraobserver variation in readings is likely to be.
Explain what is meant by interobserver variation.
Two examiners often do not give the same result. The extent to which observers agree or disagree is an important issue, whether we are considering physical examinations, laboratory tests, or other means of assessing human characteristics. We therefore need to be able to express the extent of agreement in quantitative terms.
What is the Kappa Statistic?
a kappa greater than 0.75 represents excellent agreement beyond chance, a kappa below 0.40 represents poor agreement, and a kappa in between there represents intermediate to good agreement
How do you know when to use sensitivity/specificity or kappa?
How do you validate responses from self-reports?
What is the relationship between validity and reliability?
