Decision Analysis L15

Question 1

Performance metric of classification problems

Answer 1

Misclassification Rate: Percentage of Misclassifications= 53/2000 (P(“0”|1)+ P(“1”|0))
Sensitivity: Percentage of True Defaulters who are identified = 23/32 (P(“1”|1))
Specificity: Percentage of non-defaulters correctly identified = 1924/1968 (P(“0”|0))

Question 2

Classification Problems- Confusion Matrix – Error Tradeoffs
* Recall the simple assignment rule: If p(X)>1/2, we classify it as 1 and otherwise we classify it to 0.
* Byreducingtheassignmentthresholdfrom1⁄2,wewould be able to ‘catch’ more defaulters (i.e improve sensitivity) but this would be at the expense of specificity.
* Thisisthereforeatradeoffwithoutasimple‘optimal’level and optimization depends on the context.
* We saw that it’s better to use a lower threshold for the credit default problem
* IfthegovernorofIstanbulistryingtopredictasnow storm, it makes more sense to take a low threshold.
* If a medical test is for a serious issue, it also makes sense to choose a low threshold.

Answer 2

slide 52