Biostats_5_Sensitivity, Specificity, Predictive value, Screening tests

Question 1

While analyzing the distributions for an appropriate cut-off value for a screening test, what would cause the distribution curves to be tighter? What would be the impact on the sensitivity and specificity?

Answer 1

Increasing the sample size will make the distributions tighter and alter both the sensitivity and specificity. The blue curves have a smaller sample size when comparing the red curves, and appear to improve the accuracy by tightening the distribution. This can be done with a larger sample size. The main effect is seen with the tails crossing the threshold, represented by “X’s” (to the right indicates false positives and the left negatives). A decrease in their size occurs when analyzing the area under the curve while comparing the blue curve to the red curve. Therefore, the red curves are associated with higher sensitivity and specificity because the magnitudes of false positive and false negative values are reduced.

Question 2

What effects occur when the threshold is decreased (towards point A) ?

Answer 2

The amount of false negative values decrease, leading to an increase in sensitivity.

vvvvvvvvvvvv

The amount of false positive values increase, leading to a decrease in specificity.

vvvvvvvvvvvv

The positive predictive value will decrease, but the negative predictive value will increase.

Question 3

What effects occur when the threshold is increased (towards point B) ?

Answer 3

The amount of false negative values increase, leading to a decrease in sensitivity.

vvvvvvvvvvvv

The amount of false positive values decrease, leading to an increase in specificity.

vvvvvvvvvvvv

The positive predictive value will increase, but the negative predictive value will decrease.

Question 4

Point “A” represents 100% ________ ?

Answer 4

100% sensitivity cutoff value

Question 5

How is sensitivity measured?

Answer 5

TP / ( TP + FN )

Question 6

Another name for sensitivity is the _______ rate

Answer 6

Sensitivity = True-positive rate = 1 – FN rate

This is because sensitivity is the proportion of all people with disease who test positive, or the ability of a test to correctly identify those with the disease.

Question 7

If the sensitivity is high, then the _______ are low

Answer 7

If the sensitivity is high, then the false negatives are low

Question 8

Point “B” represents 100% ________ ?

Answer 8

100% specificity cutoff value

Question 9

How is specificity measured?

Answer 9

TN / ( TN + FP )

Question 10

Another name for specificity is the _______ rate

Answer 10

Specificity = True-negative rate = 1 – FP rate

This is because specificity meansures the proportion of all people without disease who test negative, or the ability of a test to correctly identify those without the disease.

Question 11

If the specificity is high, then the _______ are low

Answer 11

If the specificity is high, then the false positives are low

Question 12

In a population of 100,000 with a disease prevalence of 1%, what are the false positives if the test specificity is 95 % ?

Answer 12

Specificity represents the probability of testing negative in patients without the disease.

vvvvvvvvvvvvvv

In this population of 100,000 people, there are 1,000 people with the disease (100,000 x .01) based on the prevalence. Therefore 99,000 people are free of the disease. If the test has a specificity of 95% then the test would be negative in 95% of these people, which is 94,050 (99,000 x .95). The amount of false positives are found in the remaining 4,950 people (99,000 x.05 or 99.000 - 94,050).

vvvvvvvvvvvvvv

A test with high specificity is typically used as a confirmatory test

vvvvvvvvvvvvvv

A positive test result on a highly specific test would rule in the disease
(SpIN = Specificity, Positive, rule In).

Question 13

In a population of 100,000 with a disease prevalence of 1%, what are the false negatives if the test sensitivity is 90 % ?

Answer 13

Sensitivity represents the probability of testing positive in patients with the disease.

vvvvvvvvvvvvvv

In this population of 100,000 people, there are 1,000 people with the disease (100,000 x .01) based on the prevalence. Therefore 900 people with the disease will test test positive if the test has a sensitivity of 90% (1,000 x .90). The amount of false negatives are found in the remaining 100 people (1,000 x.10 or 1.000 - 900).

vvvvvvvvvvvvvv

A test with high sensitivity is typically used as a screening test

vvvvvvvvvvvvvv

A negative test result on a highly sensitive test would rule the disease out
(SnOUT = Sensitivity, negative, rule out).

Question 14

What are used to determine how much a test can change the post-test probability of disease?

Answer 14

likelihood ratios

Question 15

What are likelihood ratios?

Answer 15

The likelihood of a specific test result in a patient with the target disorder compared to the likelihood of the same result in a patient without the target disorder is the LR.

vvvvvvvvvvvvvvvv

Likelihood ratios describe how the probability of a disease changes based on a diagnostic test result. They are derived from the test’s sensitivity and specificity and are used in conjunction with the pre-test probability to determine the post-test probability via Bayes’ theorem. The likelihood ratio for a positive test result (LR⁺) tells you how much the odds of the disease increase with a positive test result, while the likelihood ratio for a negative test result (LR⁻) indicates how much the odds decrease with a negative test result.

Question 16

LAs are used in clinical practice to:

Answer 16

• Use the change in post-test probability to decide whether to do the test
• Compare different tests to choose which test is most helpful in making a diagnosis

Question 17

Is there any relationship between the disease prevalence and the likelihood ratio?

Answer 17

Unlike predictive values, the likelihood ratio is independent of disease prevalence, yet, when the prevalence of the disease is very low, a positive test result is more likely to be a false positive, which is important to note because false positives following testing could lead to unnecessary testing while not changing the management.

Question 18

What is the true relationship between the likelihood ratios and disease prevalence?

Answer 18

Likelihood ratios (LRs) are not directly impacted by the prevalence of disease. LRs are properties of a diagnostic test itself, derived from its sensitivity and specificity, which are independent of disease prevalence. However, the interpretation of a test result using LRs is influenced by the pre-test probability, which is often estimated based on disease prevalence in the population being studied.

Question 19

What is the relevance of the positive likelihood ratio ?

Answer 19

The likelihood ratio is an indicator of test performance.

vvvvvvvvvvvvvv

The positive likelihood ratio is calculated by:
dividing sensitivity by (1-specificity).

vvvvvvvvvvvvvv

For example, a positive likelihood ratio of 9 indicates that a positive test result is seen 9 times more frequently in patients with the disease than in patients without the disease.

Question 20

A +LR of >10 increases post-test probability by

Answer 20

45% (very useful)

vvvvvvvvvvvvvvvv

LR+ > 10 indicates a highly specific test

Question 21

A +LR of 5 increases post-test probability by

Answer 21

30% (moderately useful)

Question 22

A +LR of 2 increases post-test probability by

Answer 22

15% (less useful)

Question 23

A +LR of 1 increases post-test probability by

Answer 23

Zero (useless)

Question 24

What is the relevance of the negative likelihood ratio ?

Answer 24

The negative likelihood ratio (LR⁻) is a critical diagnostic tool used to quantify how much a negative test result reduces the likelihood of a disease. It reflects the probability of a negative test result in patients with the disease compared to those without it. A lower LR⁻ indicates a stronger ability of the test to rule out disease.

vvvvvvvvvvvvvv

The negative likelihood ratio is calculated by:
dividing (1 - sensitivity) by specificity.

Question 25

LR⁻ < 0.1:

Answer 25

This is considered very useful and suggests the test strongly reduces the post-test probability of disease (by approximately 45%). It is excellent for ruling out disease.

vvvvvvvvvvvvvv

LR– < 0.1 indicates a highly sensitive test.

Question 26

LR⁻ between 0.1 and 0.2:

Answer 26

Moderately useful; decreases the post-test probability of disease by about 30-45%.

Question 27

LR⁻ between 0.2 and 0.5:

Answer 27

Less useful; decreases the post-test probability by about 15-30%.

Question 28

LR⁻ > 0.5:

Answer 28

Clinically unhelpful, as it does not significantly reduce the likelihood of disease.

Question 29

How are the pre-test odds linked to the post-test odds?

Answer 29

Pre-test odds x LR = Post-test odds

vvvvvvvvvvvvvv

Directly proportional (the higher the LR or pre-test odds, then the more likely the post-test odds)

Question 30

How is the post-test probability calculated?

Answer 30

post-test odds / (post-test odds + 1) = Posttest probability

Question 31

What is on the Y-axis in a receiver operating characteristic (ROC) curve?

Answer 31

true-positive rate (sensitivity)

Question 32

What is on the X-axis in a receiver operating characteristic (ROC) curve?

Answer 32

false-positive rate (1 – specificity)

Question 33

ROC curve demonstrates … ?

Answer 33

how well a diagnostic test can distinguish between 2 groups

(eg, disease vs healthy).

Question 34

What does curve “A” represent?

Answer 34

The area under curve “A” is near 1 and the test is very accurate.

Question 35

What does curve “B” represent?

Answer 35

The area under curve “B” is near 1/2 and the test lacks predictive value.

Question 36

What is the purpose of this two receiver operating characteristic (ROC) curve?

Answer 36

A receiver operating characteristic (ROC) curve iillustrates the tradeoff between sensitivity and specificity which is made when choosing a cutoff value for positive and negative test results. The area under ROC represents accuracy of the test (the number of true positives plus true negatives divided by the number of all observations). An accurate test would have area under the ROC close to 1.0 (rectangular shape) whereas a test with no predictive value would be represented by a straight line.

vvvvvvvvvvvvvv

For example, a cuttoff at point “X” would have a high sensitivity and low specificity, a cutoff a point “Y” would have low sensitivity and high specificity. “X” is closer to 100% sensitivity than to point “Y” and “Y” is closer to 100% specificity than to point “X”. Based on these observations, it can be concluded that “X” would require a lower serum marker for a positive test result.

Question 37

For a receiver operating characteristic (ROC) curve, the better performing tests will have … ?

Answer 37

a higher area under the curve (AUC), with the curve closer to the upper left corner.

Question 38

What is the calculation for the positive predicitve value?

Answer 38

PPV = TP / (TP + FP)

vvvvvvvvvvvvvv

Probability that a person who has a positive test result actually has the disease.

Question 39

Does the prevalence impact predicitve values?

Answer 39

Yes. the PPV varies directly with prevalence while the NPV varies indirectly with prevalence.

vvvvvvvvvvvvvv

A high prevalence corresponds to a positive test more likely being a true positive (PPV is high).

vvvvvvvvvvvvvv

A low prevalence corresponds to a negative test more likely to be a true negative (NPV is high).

Question 40

How does a high pretest probability impact the positive predicitve value?

Answer 40

PPV varies directly with pretest probability (baseline risk, such as prevalence of disease).

vvvvvvvvvvvvvv

A high pretest probability corresponds to a high PPV.

vvvvvvvvvvvvvv

For example, you test two patients with HIV. The first patient has many risk factors, the seond patient doesn’t. The first patient will have a higher PPV with a positive test result because the pretest probability is higher. The Second patient without risk factors will not have such a high PPV because the pretest probability is not as high as the the first patient.

Question 41

What is the calculation for the negative predicitve value?

Answer 41

NPV = TN / (TN + FN)

vvvvvvvvvvvvvv

Probability that a person with a negative test result actually does not have the disease.

Question 42

Test efficiency is ….

Answer 42

(TP + TN)/(TP + FN + FP + TN)

Question 43

Prevalence based on TN and FN is …

Answer 43

Prevalence = [(TP + FN) / (TP + FN + FP + TN)]