Chapter 5- Experimental Methods 2- ROC Analysis

Question 1

who was one of the earliest adopters of ROC graphs in ML?

Answer 1

Spackman (1989)

Question 2

FP rate =

Answer 2

FP / N

Question 3

TP rate =

Answer 3

TP / P

Question 4

precision =

Answer 4

TP / (TP + FP)

Question 5

positive predictive value =

Answer 5

precision = TP / (TP + FP)

Question 6

recall =

Answer 6

TP / P

Question 7

accuracy =

Answer 7

(TP + TN) / P + N

Question 8

sensitivity =

Answer 8

recall = TP / P

Question 9

specificity =

Answer 9

TN / (FP + TN)

Question 10

F-measure =

Answer 10

2 / (1/precision)+(1/recall)

precision = TP / (TP + FP)
recall = TP / P

Question 11

what does a roc graph plot (x and y)?

Answer 11

x axis = fp rate

y axis = tp rate

Question 12

what is on the x axis of a roc graph?

Answer 12

fp rate

Question 13

what is on the y axis of a roc graph?

Answer 13

tp rate

Question 14

what is a discrete classifier?

Answer 14

outputs only a class label

Question 15

what does point (0,0) represent on a roc curve?

Answer 15

never issuing a positive classification

Question 16

what does point (1,1) represent on a roc curve?

Answer 16

unconditionally issuing positive classifications

Question 17

what does point (0,1) represent on a roc curve?

Answer 17

perfect classification

Question 18

one point on a roc curve is better than another if…?

Answer 18

it is northwest

Question 19

where would a conservative classifier appear on the roc graph?

Answer 19

on the left hand side

makes positive classifications only with strong evidence

Question 20

where would a liberal classifier appear on the roc graph?

Answer 20

on the upper right hand side

makes positive classifications with weak evidence

Question 21

what does y=x represent on a roc curve?

Answer 21

random performance

Question 22

what can we say about a classifier at point (0.7,0.7) on a roc graph?

Answer 22

it acts randomly, guessing the positive class 70% of the time

Question 23

what do we say about classifiers appearing in the lower right triangle of the roc graph?

Answer 23

it has useful information but is applying it wrong. we can simply negate the output