Developing a new test

Question 1

ACCE model

Answer 1

Four criteria for evaluating a new test

1. Analytical validity- e.g. Technical test performance, reproducibility, sensitivity and specificity
2. Clinical validity- e.g. Does the test detect or predict clinical outcomes of interest, has the test been validated on all populations which it may be offered
3. Clinical utility- e.g. What impact does this have on patient care?
4. ESLI- e.g. Ethical, legal and social implications of implementing the test All

Question 2

All new tests must be validated and verified before being used clinically… What is validation and verification?

Answer 2

Validation- Are we building the right thing? i.e. measures that a product or system meets the needs of the customer and other identified stakeholders (external- users)

Verification- Are we building the thing right? i.e. measures if the product or system complies with the requirements or specification (internal)

Question 3

What is sensitivity and how is it calculated?

Answer 3

(True positive rate) Measures the proportion of actual positives (sick people) that have been correctly identified by the test
TP
Sensitivity = ——————- x 100
(TP + FN)

Question 4

What is sensitivity and how it is calculated?

Answer 4

(True negative rate) Measures the proportion of actual negatives that have been correctly identified as such.
TN
Specificity = ————————- x 100
(TN + FP)

Question 5

What is Positive predictive value (PPV) and how is it calculated?

Answer 5

The probability that subjects with a positive screening test truly have the disease
TP
PPV = —————————— x 100
(TP + FP)

Question 6

What is Negative predictive value (NPV) and how is it calculated?

Answer 6

The probability that subjects with a negative screening test truly do not have the disease
TN
NPV = ——————————- x 100
(FN + TN)

Question 7

What is accuracy and how is it calculated?

Answer 7

How often a test gives the correct results
(TP + TN)
Accuracy = —————————————— x 100
(TP + TN + FP + FN)

Question 8

What is the definition of precision?

Answer 8

The degree to which separate measurements differ, which indicates how well a single test result is representative of a number of repeats.

Can be expressed as the standard deviation of a set of replicate results or as a confidence interval around the mean result

Question 9

What is the definition of trueness?

Answer 9

For quantitative test, where results can have any value between two limits, this is a measurement of how close the test result is to the reference value.

Question 10

What is the definition of reproducibility?

Answer 10

The closeness of results obtained when using the same samples under different test conditions

Question 11

What is the definition of repeatability?

Answer 11

The closeness of agreement of results obtained when using the same samples under the same test conditions and repeated over a short period

Question 12

What is robustness of a test?

Answer 12

Reliability of a test, in test terms of how well it maintains precision, when certain variables (inc test conditions) are changed e.g. changing technician or instrument

Question 13

What is the limit of detection?

Answer 13

For quantitative tests, where the results can have any value between two limits, this is the lowest quantity of analyte that can be reliably detected by the test