Chapter 7 - Decision Analytic Thinking 1: What is a good model? Flashcards
What the deal w/ classification accuracy?
It is an easy to measure metric to asses the models perfomance. However, it is to simplistic to use it in real lfie.
Accuracy = N of correct decisions made/ Total N decisions made = 1 - error rate
What is the deal w/ the Confusion Matrix
Used to decompose and count the different types of correct and incorrect devisions made by a classiffier. Used for True Positive and False Positives.
What is a skewed class distribution? What is the problem w/ it?
This is an unbalanced class distribution in which one class different than the other. The problem is: 1. More skewness leads to a break down in evaluation based on accuracy.
What is the problem w/ accuracy?
- Accuracy is misleading when there is skewness.
2. It makes no distinction between False Positive and False Negatives.
What is the deal w/ the Expected Value Framework?
Calculates the weighted average of the values of the different possible outcomes.
It decomposes data-analytic thinking into:
1. The Structure of the problem
2. The elements of the analysis that can be extracted from the data.
3. The elements of the analysis that need to be acquired from other sources.
Why is the Expected Value Framework used?
It is used to calculate the Expected payoff of a model. To know whether the model should be used, the EV of profits > EV costs. This all depends on the Vr and Vnr.
What is the benefit of using the Expected Value Framework instead of using Accuracies?
The EFV can be used to compare two different models
Explain the majority classifier?
This is a naive classifier that always chooses the majority class of the training data as baseline.
What is a decision stump and what is it used for?
This is a decision tree w/ only one internal node, the root node. This means that only the single most informative feature is selected to make a decision.
