Biostats_4_Sensitivity, Specificity, Predictive value, Screening tests

Question 1

While analyzing the distributions for an appropriate cut-off value for a screening test, what would cause the distribution curves to be tighter? What would be the impact on the sensitivity and specificity?

Answer 1

Increasing the sample size will make the distributions tighter and alter both the sensitivity and specificity. The blue curves have a smaller sample size when comparing the red curves, and appear to improve the accuracy by tightening the distribution. This can be done with a larger sample size. The main effect is seen with the tails crossing the threshold, represented by “X’s” (to the right indicates false positives and the left negatives). A decrease in their size occurs when analyzing the area under the curve while comparing the blue curve to the red curve. Therefore, the red curves are associated with higher sensitivity and specificity because the magnitudes of false positive and false negative values are reduced.

Question 2

What effects occur when the threshold is decreased (towards point A) ?

Answer 2

The amount of false negative values decrease, leading to an increase in sensitivity.

vvvvvvvvvvvv

The amount of false positive values increase, leading to a decrease in specificity.

vvvvvvvvvvvv

The positive predictive value will decrease, but the negative predictive value will increase.

Question 3

What effects occur when the threshold is increased (towards point B) ?

Answer 3

The amount of false negative values increase, leading to a decrease in sensitivity.

vvvvvvvvvvvv

The amount of false positive values decrease, leading to an increase in specificity.

vvvvvvvvvvvv

The positive predictive value will increase, but the negative predictive value will decrease.

Question 4

Point “A” represents 100% ________ ?

Answer 4

100% sensitivity cutoff value

Question 5

How is sensitivity measured?

Answer 5

TP / ( TP + FN )

Question 6

Another name for sensitivity is the _______ rate

Answer 6

Sensitivity = True-positive rate = 1 – FN rate

This is because sensitivity is the proportion of all people with disease who test positive, or the ability of a test to correctly identify those with the disease. Sensitivity represents the probability of testing positive in patients with the disease.

Question 7

If the sensitivity is high, then the _______ are low

Answer 7

If the sensitivity is high, then the false negatives are low.

Question 8

A highly sensitive test would rule ________ the disease

Answer 8

“OUT”

vvvvvvvvvvvvvvv

A negative test result on a highly sensitive test rules the disease out.

vvvvvvvvvvvvvvv

SnOUT = Sensitivity, negative, rule out

Question 9

A test with high sensitivity is typically used as a __________ test

Answer 9

screening test

Question 10

In a population of 100,000 with a disease prevalence of 1%, what are the false negatives if the test sensitivity is 90 % ?

Answer 10

In this population of 100,000 people, there are 1,000 people with the disease (100,000 x .01) based on the prevalence. Therefore 900 people with the disease will test test positive if the test has a sensitivity of 90% (1,000 x .90). The amount of false negatives are found in the remaining 100 people (1,000 x.10 or 1.000 - 900).

Question 11

Point “B” represents 100% ________ ?

Answer 11

100% specificity cutoff value

Question 12

How is specificity measured?

Answer 12

TN / ( TN + FP )

Question 13

Another name for specificity is the _______ rate

Answer 13

Specificity = True-negative rate = 1 – FP rate

This is because specificity measures the proportion of all people without disease who test negative, or the ability of a test to correctly identify those without the disease. Specificity represents the probability of testing negative in patients without the disease.

Question 14

If the specificity is high, then the _______ are low

Answer 14

If the specificity is high, then the false positives are low.

Question 15

A highly specific test will rule a positive test _____ .

Answer 15

“IN”

vvvvvvvvvvvvvv

A positive test result on a highly specific test would rule in the disease

vvvvvvvvvvvvvv

SpIN = Specificity, Positive, rule In

Question 16

In a population of 100,000 with a disease prevalence of 1%, what are the false positives if the test specificity is 95 % ?

Answer 16

In this population of 100,000 people, there are 1,000 people with the disease (100,000 x .01) based on the prevalence. Therefore 99,000 people are free of the disease. If the test has a specificity of 95% then the test would be negative in 95% of these people, which is 94,050 (99,000 x .95). The amount of false positives are found in the remaining 4,950 people (99,000 x.05 or 99.000 - 94,050).

Question 17

A test with high specificity is typically used as a ______________ test.

Answer 17

confirmatory test.

Question 18

What are used to determine how much a test can change the post-test probability of disease?

Answer 18

likelihood ratios

Question 19

What are likelihood ratios?

Answer 19

The likelihood ratio (LR) is a measure used in diagnostic testing to assess how much a specific test result changes the probability of having a target disorder. It compares the probability of a given test result in patients with the disorder (true positive rate or true negative rate) to the probability of the same test result in patients without the disorder (false positive rate or false negative rate).

Question 20

Likelihood ratios are used in clinical practice to:

Answer 20

• Use the change in post-test probability to decide whether to do the test
• Compare different tests to choose which test is most helpful in making a diagnosis

Question 21

Is there any relationship between the disease prevalence and the likelihood ratio?

Answer 21

Unlike predictive values, the likelihood ratio is independent of disease prevalence, yet, when the prevalence of the disease is very low, a positive test result is more likely to be a false positive, which is important to note because false positives following testing could lead to unnecessary testing while not changing the management.

Question 22

What is the true relationship between the likelihood ratios and disease prevalence?

Answer 22

Likelihood ratios (LRs) are not directly impacted by the prevalence of disease. LRs are properties of a diagnostic test itself, derived from its sensitivity and specificity, which are independent of disease prevalence. However, the interpretation of a test result using LRs is influenced by the pre-test probability, which is often estimated based on disease prevalence in the population being studied.

Question 23

What is the relevance of the positive likelihood ratio ?

Answer 23

The likelihood ratio is an indicator of test performance.

vvvvvvvvvvvvvv

The positive likelihood ratio is calculated by:
dividing sensitivity by (1-specificity).

vvvvvvvvvvvvvv

For example, a positive likelihood ratio of 9 indicates that a positive test result is seen 9 times more frequently in patients with the disease than in patients without the disease.

Question 24

A +LR of >10 increases post-test probability by

Answer 24

45% (very useful)

vvvvvvvvvvvvvvvv

LR+ > 10 indicates a highly specific test

Question 25

A +LR of 5 increases post-test probability by

Answer 25

30% (moderately useful)

Question 26

A +LR of 2 increases post-test probability by

Answer 26

15% (less useful)

Question 27

A +LR of 1 increases post-test probability by

Answer 27

Zero (useless)

Question 28

What is the relevance of the negative likelihood ratio ?

Answer 28

The negative likelihood ratio (LR⁻) is a critical diagnostic tool used to quantify how much a negative test result reduces the likelihood of a disease. It reflects the probability of a negative test result in patients with the disease compared to those without it. A lower LR⁻ indicates a stronger ability of the test to rule out disease.

vvvvvvvvvvvvvv

The negative likelihood ratio is calculated by:
dividing (1 - sensitivity) by specificity.

Question 29

LR⁻ < 0.1:

Answer 29

This is considered very useful and suggests the test strongly reduces the post-test probability of disease (by approximately 45%). It is excellent for ruling out disease.

vvvvvvvvvvvvvv

LR– < 0.1 indicates a highly sensitive test.

Question 30

LR⁻ between 0.1 and 0.2:

Answer 30

Moderately useful; decreases the post-test probability of disease by about 30-45%.

Question 31

LR⁻ between 0.2 and 0.5:

Answer 31

Less useful; decreases the post-test probability by about 15-30%.

Question 32

LR⁻ > 0.5:

Answer 32

Clinically unhelpful, as it does not significantly reduce the likelihood of disease.

Question 33

How are the pre-test odds linked to the post-test odds?

Answer 33

Pre-test odds x LR = Post-test odds

vvvvvvvvvvvvvv

Directly proportional (the higher the LR or pre-test odds, then the more likely the post-test odds)

Question 34

How is the post-test probability calculated?

Answer 34

post-test odds / (post-test odds + 1) = Posttest probability

Question 35

What is on the Y-axis in a receiver operating characteristic (ROC) curve?

Answer 35

true-positive rate (sensitivity)

Question 36

What is on the X-axis in a receiver operating characteristic (ROC) curve?

Answer 36

false-positive rate (1 – specificity)

Question 37

ROC curve demonstrates … ?

Answer 37

how well a diagnostic test can distinguish between 2 groups

(eg, disease vs healthy).

Question 38

What does curve “A” represent?

Answer 38

The area under curve “A” is near 1 and the test is very accurate.

Question 39

What does curve “B” represent?

Answer 39

The area under curve “B” is near 1/2 and the test lacks predictive value.

Question 40

What is the purpose of this two receiver operating characteristic (ROC) curve?

Answer 40

A receiver operating characteristic (ROC) curve iillustrates the tradeoff between sensitivity and specificity which is made when choosing a cutoff value for positive and negative test results. The area under ROC represents accuracy of the test (the number of true positives plus true negatives divided by the number of all observations). An accurate test would have area under the ROC close to 1.0 (rectangular shape) whereas a test with no predictive value would be represented by a straight line.

vvvvvvvvvvvvvv

For example, a cuttoff at point “X” would have a high sensitivity and low specificity, a cutoff a point “Y” would have low sensitivity and high specificity. “X” is closer to 100% sensitivity than to point “Y” and “Y” is closer to 100% specificity than to point “X”. Based on these observations, it can be concluded that “X” would require a lower serum marker for a positive test result.

Question 41

For a receiver operating characteristic (ROC) curve, the better performing tests will have … ?

Answer 41

a higher area under the curve (AUC), with the curve closer to the upper left corner.

Question 42

What is the calculation for the positive predicitve value?

Answer 42

PPV = TP / (TP + FP)

vvvvvvvvvvvvvv

Probability that a person who has a positive test result actually has the disease.

Question 43

Does the prevalence impact predicitve values?

Answer 43

Yes. the PPV varies directly with prevalence while the NPV varies indirectly with prevalence.

vvvvvvvvvvvvvv

A high prevalence corresponds to a positive test more likely being a true positive (PPV is high).

vvvvvvvvvvvvvv

A low prevalence corresponds to a negative test more likely to be a true negative (NPV is high).

Question 44

How does a high pretest probability impact the positive predicitve value?

Answer 44

PPV varies directly with pretest probability (baseline risk, such as prevalence of disease).

vvvvvvvvvvvvvv

A high pretest probability corresponds to a high PPV.

vvvvvvvvvvvvvv

For example, you test two patients with HIV. The first patient has many risk factors, the seond patient doesn’t. The first patient will have a higher PPV with a positive test result because the pretest probability is higher. The Second patient without risk factors will not have such a high PPV because the pretest probability is not as high as the the first patient.

Question 45

What is the calculation for the negative predicitve value?

Answer 45

NPV = TN / (TN + FN)

vvvvvvvvvvvvvv

Probability that a person with a negative test result actually does not have the disease.

Question 46

Test efficiency is ….

Answer 46

(TP + TN)/(TP + FN + FP + TN)

Question 47

Prevalence based on TN and FN is …

Answer 47

Prevalence = [(TP + FN) / (TP + FN + FP + TN)]