Indices Of Accuracy Flashcards
What are the three important commonly used indices? And their statistical properties?
AUC, partial area, TPF (FPF) These indices can be estimated both parametrically and nonparametrically
AUC do and interpreted and is based on?
The area under the curve (AUC), as a one- dimensional (uni-dimensional) index, summarizes the “overall” location of the entire ROC curve. It is of great interest, since it has a meaningful interpretation. The AUC can be interpreted as the probability that a randomly chosen diseased subject is rated or ranked as more likely to be diseased than a randomly chosen nondiseased subject (5). This interpretation is based on nonparametric Mann-Whitney U statistics that is used in calculating AUC
What is two tests are to be compared?
desirable to compare the entire ROC curve rather than at a particular point (1).
What does AUC=1 mean? And AUC=0.5 or 0?
The maximum AUC=1 means that the diagnostic test is perfect in the differentiation between the diseased and nondiseased. This happens when the
distribution of test results for the diseased and nondiseased do not overlap. AUC =0.5 means the chance discrimination that curve located on diagonal line in ROC space. The minimum AUC should be considered a chance level i.e. AUC=0.5 while AUC=0 means test incorrectly classify all subjects with diseased as negative and all subjects with nondiseased as positive that is extremely unlikely to happen in clinical practice.
Partial area index?
it may still be argued that a large part of the area arises from the right side of the unit square where the high false positive fraction is of no clinical relevance. Thus, one may be adding noise when using the area index to compare the two different diagnostic systems. Also, two ROC curves with the same area may cross, but one may have higher TPF than another in the clinically relevant range of FPF. In this situation, a partial area under the curve corresponding to a clinical relevant FPF is recommended as an index of choice
TPF (FPF) index?
One may be interested in comparing the performance of two tests at a given FPF for clinical reasons, especially in a case where two ROC curves cross. The areas under the curves may be equal but in a
clinical range of interest the FPF for one test may be superior to that of the other. Also, TPF(FPF) can easily be applied and it is readily understood. However, the main criticism of TPF(FPF) as an index of accuracy is that the different investigators may not report the estimates of TPF at the same FPF. If so, the two TPF from the different investigators of the same test may not be comparable.
