Quiz 3

Question 1

three main types of outcomes of interest

Answer 1

predicted numerical value
predicted class membership
propensity (probability when categorical)

Question 2

predicting class membership

Answer 2

classification

Question 3

new records most likely to be part of class

Answer 3

ranking

Question 4

predictive accuracy measures

Answer 4

mean absolute error/deviation, average error, mean absolute percentage error, root mean squared error, total sum of squared errors

Question 5

compares the model’s predictive performance to a baseline model that has no predictors

Answer 5

lift chart

Question 6

what is a lift chart looking for?

Answer 6

subset of records that has the highest cumulative predicted values

Question 7

observation that belongs to one class but model put it in another

Answer 7

misclassification error

Question 8

summarizes the correct and incorrect classifications that a classified produced for a certain dataset (validation)

Answer 8

classification/confusion matrix

Question 9

overall error rate

Answer 9

incorrect/total

Question 10

error

Answer 10

actual - prediction

Question 11

ability to correctly detect all important class members

Answer 11

sensitivity of classifier

Question 12

ability to rule out negative class members correctly

Answer 12

specificity of classifier

Question 13

plots 1 - sensitivity and specificity

Answer 13

ROC curve

Question 14

ability to detect only the important class members

Answer 14

precision of classifier

Question 15

average cost of miscalculation per classified observation

Answer 15

average misclassification cost

Question 16

score model to validation set that is random or score model to an oversampled validation set and reweight the results to remove the effects of oversampling

Answer 16

how to adjust for oversampling

Question 17

how many responders from whole data is a sample responder worth

Answer 17

oversampling weights

Question 18

accurately classify the most interesting/important cases

Answer 18

goal of ranking

Question 19

actual is no, predicted is no (0,0)

Answer 19

true negative

Question 20

actual is yes, predicted is yes (1,1)

Answer 20

true positive

Question 21

actual is yes, predicted is no (1,0)

Answer 21

false negative

Question 22

actual is no, predicted is yes (0,1)

Answer 22

false positive

Question 23

accuracy

Answer 23

1 - error

Question 24

error with confusion matrix

Answer 24

(false negative + false positive) / total

Question 25

if it is important to predict positive values correctly what should you do?

Answer 25

lower the cut-off

Question 26

sensitivity

Answer 26

true positive / (true positive + false negative)

Question 27

F1

Answer 27

(2 * precision * recall) / (precision + recall)

Question 28

what is the ideal part of a ROC curve?

Answer 28

top lift, high sensitivity and high specificity