Chapter 5- Experimental Methods 2- ROC Analysis Flashcards
who was one of the earliest adopters of ROC graphs in ML?
Spackman (1989)
FP rate =
FP / N
TP rate =
TP / P
precision =
TP / (TP + FP)
positive predictive value =
precision = TP / (TP + FP)
recall =
TP / P
accuracy =
(TP + TN) / P + N
sensitivity =
recall = TP / P
specificity =
TN / (FP + TN)
F-measure =
2 / (1/precision)+(1/recall)
precision = TP / (TP + FP) recall = TP / P
what does a roc graph plot (x and y)?
x axis = fp rate
y axis = tp rate
what is on the x axis of a roc graph?
fp rate
what is on the y axis of a roc graph?
tp rate
what is a discrete classifier?
outputs only a class label
what does point (0,0) represent on a roc curve?
never issuing a positive classification
what does point (1,1) represent on a roc curve?
unconditionally issuing positive classifications
what does point (0,1) represent on a roc curve?
perfect classification
one point on a roc curve is better than another if…?
it is northwest
where would a conservative classifier appear on the roc graph?
on the left hand side
makes positive classifications only with strong evidence
where would a liberal classifier appear on the roc graph?
on the upper right hand side
makes positive classifications with weak evidence
what does y=x represent on a roc curve?
random performance
what can we say about a classifier at point (0.7,0.7) on a roc graph?
it acts randomly, guessing the positive class 70% of the time
what do we say about classifiers appearing in the lower right triangle of the roc graph?
it has useful information but is applying it wrong. we can simply negate the output
