4. Targeting Marketing Interventions

Question 1

Confusion Matrix

Answer 1

A confusion matrix allows visualization of the performance of a model

• Accuracy
  
  • (correct prediction) / (all predictions)
  • (TP + TN) / (TP + TN + FP + FN)
• Error rate
  
  • (false predictions) / (all predictions)
  • ​​(FP + FN) / (TP + TN + FP + FN)
• TP rate
  
  • (positive labels) / (all positives)
  • ​TP / (TP + FN)
• FP rate
  
  • (false positives) / (all negatives)
  • ​FP / (FP + TN)

Question 2

Receiver Operating Characteristic (ROC)

Answer 2

• The ROC graph shows the entire space of performance possibilities for a given model, independent of class balance
• It depicts relative trade-offs that a classifier makes between benefits (true positives) and costs (false negatives)
• The line ranking (0,0) to (1,1) is the strategy of guessing randomly

Question 3

Area under the curve (AUC)

Answer 3

• AUC is the probability that the model will rank a randomly chosen positive case higher than a negative case
• AUC is useful when a single number is needed to summarize performance, or when nothing is known about the operating conditions

Question 4

Expected Value Framework

Answer 4

The expected value framework is an analytical tool that is extremely helpful in organizing thinking about data-analytic problems.

Combines:

• Structure of the problem
• Elements of the analysis that can be extracted from the data
• Elements of the analysis that need to be acquired from business knowledge

Question 5

Base Scenario

Answer 5

A base scenario is needed to compare the outcomes of the expected value framework.

Question 6

Benefit/Cost Matrix

Answer 6

• Summarizes the benefits and costs of each potential outcome
• Always comparing with a base scenario

Question 7

Expected Profit

Answer 7

By multiplying the costs and benefits of the cost/benefit matrix with the number of observations in the confusion matrix we can calculate the expected profit.