Stats

Question 1

Define sensitivity / specificity / positive predictive value / negative predictive value of a test

Answer 1

• Sensitivity = probability that a given test will correctly identify those animals with the disease = ability of a test or instrument to yield a positive result for a subject that has that disease
  (positive test => disease)
• Specificity = probability that a given test will identify those animals that do not have the disease = ability of the test to obtain negative results for a subject who does not have a disease
  (negative test => no disease)
• PPV = probability that an animal with a positive test result actually has the condition
  (disease => positive test)
• NPV = probability that an animal with a negative test result is actually free of the condition
  (no disease => negative test)

Question 2

Define the accuracy of a test

Answer 2

The accuracy of a test is its ability to differentiate the sick and healthy cases correctly

Question 3

Formula for Se, Sp, PPV, NPV

Answer 3

• Sensitivity = True positives / All positives = True positives / (True positives + False negatives)
• Specificity= True negatives / All negatives = True negatives / (True negatives + False positives)
• PPV = True Positives / (True positives + False positives)
• NPV = True negatives / (True negatives + False negatives)

Question 4

Formula for the accuracy of a binary test

Answer 4

Accuracy = (True positive + True negative) / (True positive + False positive + True negative + False negative)

Question 5

What parameter (independent of the test quality) influences PPV and NPV

Answer 5

Disease prevalence in the studied population

Tests better at “ruling in” a disease than ruling it out when prevalence is higher -> higher prevalence means higher PPV, lower prevalence means higher NPV

Question 6

Define the Positive likelihood ratio and Negative likelihood ratio

Answer 6

Positive likelihood ratio = probability that a positive test would be expected in a patient who has the disease divided by the probability that a positive test would be expected in a patient without a disease = true positive rate / false positive rate
If LR+ > 1 the test is more likely to be positive in patients with the disease ; good tests have LR+ >10

Negative likelihood ratio = the probability of a patient testing negative who has a disease divided by the probability of a patient testing negative who does not have a disease = false negative rate / true negative rate
If LR- < 1 the test is more likely to be negative in patients without the disease ; good tests have LR- < 0.1

Question 7

Formula for positive likelihood ratio and negative likelihood ratio

Answer 7

Positive Likelihood Ratio = Sensitivity / (1-Specificity)

Negative Likelihood Ratio = (1- Sensitivity) / Specificity

Question 8

Formula for diagnostic odds ratio (DOR)

Answer 8

DOR = (True positive / false negative) / (False positive / true negative) = (True positive x True negative) / (False negative * False positive)

Question 9

A test gave the following results:
- 1,000 individuals tested
- 427 had positive results ; 369 of them had the disease
- 573 had negative results ; 558 of them did not have the disease

Calculate Se, Sp, PPV, NPV, positive likelihood ratio, negative likelihood ratio

Answer 9

• Sensitivity = 369/384 = 0.96
• Specificity = 558/616 = 0.90
• PPV = 369/427 = 0.86
• NPV = 558 /573 = 0.97
• Positive Likelihood Ratio = 0.96/(1-0.90) = 10.2
• Negative Likelihood Ratio = (1- 0.96)/0.90 = 0.043

Question 10

What is the AUROC of a test

Answer 10

Area under the receiver operator characteristic: plot of sensitivity versus 1 - specificity

Used to determine the ability of a test or score to predict diagnosis / outcome. The y-axis is true-positive proportion and the x-axis is false positive proportion.

-> indicates overall accuracy of a test

Usually a test is acceptable if AUROC is 0.8-1.0. (1.0 is perfect, 0.5 shows no predictive value). AUROC of 0.85 = 85% accuracy of the test

Question 11

What is the calibration graph of a score

Answer 11

Graph indicating the predicted negative outcome risk (predicted by the score) and the measured negative outcome risk (outcome that truly happened) against score values. If the score is well calibrated the predicted and measured should match.

Question 12

What accuracy parameters of a test are affected / not affected by the prevalence of the disease

Answer 12

• Affected: NPV, PPV, accuracy
• Not affected: Se, Sp, positive likelihood ratio, negative likelihood ratio, diagnostic odds ratio

Question 13

When can a test’s accuracy be evaluated using Sp / Se

Answer 13

If the test is binary (positive / negative) -> for continuous values, would have to select a threshold

If the test is continuous, can use the AUROC

Question 14

To design a screening test, is Se or Sp more important? For a confirmatory test?

Answer 14

Screening -> need good Se (all patients that have the disease need to test positive)

Confirmatory -> need good Sp (patients that don’t have the disease need to test negative)

Question 15

What is the difference between accuracy and precision of a test

Answer 15

• Accuracy = how close a given set of observations are to their true value (measure of systematic errors = bias)
• Precision = how close the observations are to each other (measure of random errors = variability)

Question 16

What is the p-value

Answer 16

p-value = probability that the null hypothesis is true -> null hypothesis can be rejected if p < 0.05.
Usually null hypothesis = there is no difference between groups. If p<0.05, this can be rejected -> means there is a difference between groups

Question 17

What is Younden’s index

Answer 17

An index of overall accuracy of a test

Younden’s index = Se + Sp -1

(in a perfect test, Younden’s index = 1)

Question 18

Name a statistical test than can be used to compare values between different groups (with normally distributed data)

Answer 18

ANOVA test
(T-test if only 2 groups)

Question 19

Name a statistical test than can be used to determine if 2 variables are correlated (with normally distributed data)

Answer 19

Pearson correlation test

Can also use a logistic regression

Question 20

Name one test that can test for normality of data distribution

Answer 20

Shapiro-Wilk test

Question 21

What is a type I / type II error in statistics

Answer 21

• Type I error = finding that a difference is significant (= rejecting the null hypothesis) when there is not an actual difference
• Type II error = not finding a significant difference when it does exist (often due to under-powered study)