L4: Classification Tasks Flashcards
After this deck: - Apply linear models to solving different classification problems; - Assess the accuracy of coefficient estimates and the accuracy of the model; - Produce analysis on the model output.;
To predict whether or not an individual will default on his or her credit card payment, on the basis of gender, education, age, history of past payment, etc.
This is an example of what kind of task?
Classification: we are trying to predict a categorical response variable, Y given our feature vector X
Why can’t we use linear regression for classification tasks?
The linear regression model would produce probabilities less than 0 or larger than 1 when computing the likelihood of say, a defaulting borrower.
What is an alternative model that can be used for classification tasks?
The logistic function can be used to predict the likelihood of a binary category, and hence determine classes.
In logistic regression models, how does the coefficient alter the resulting model?
Unlike in linear regression, the coefficient affects the probability of the resulting category, not the category itself.
The coefficient affects the shape of the corresponding model, if positive (left), increasing the X will increase the Pr(Y=1). If negative (right), increasing the X will decrease the Pr(Y=1)
What is the statistic used to assess the accuracy of the coefficients?
The z-statistic.
Using the logistic regression formula, what is the estimated probability of default for a person with balance of $1000?
P(default=yes | balance = 1000) =
[e-10.65 + 0.0055 x 1000] / [1 + e-10.65 + 0.0055 x 1000] = 0.00576
How are the regression coefficients estimated in logistic regression?
Maximum likelihood is used. We find the maximum likelihood of the data, given the two coefficients (B0,B1).
That is, seek the B0 and B1 such that the predicted probability for each X corresponds as closely to the observed chance of Y
What is Linear Discriminant Analysis?
It is a classification method that utilises generative modelling. We attempt to estimate the within-class density of X, given the class label. We use Bayes’ formula and the prior probability to estimate the posterior probability of Y.
How does LDA differ from logistic regression?
LDA does not suffer from the unstable estimates of the classes that LogReg does.
Its decision boundaries between classes also mean better accuracy.
What is the misclassification rate?
What is the False Positive Rate?
What is the True Positive Rate?
With classification threshold = 0.5, we receive 23+252 / 10000 errors a 2.75% misclassification rate!
FPR = FP / [FP+TN]
The fraction of negative samples that are mis-classified
TPR = TP / [TP + FN]
The fraction of positive samples that are correctly classified
What is the misclassification rate?
With classification threshold = 0.5, we receive 23+252 / 10000 errors a 2.75% misclassification rate!
What is the misclassification rate?
With classification threshold = 0.5, we receive 23+252 / 10000 errors a 2.75% misclassification rate!
What is Quadratic Discriminant Analysis?
QDA is a variant of LDA in which an individual covariance matrix is estimated for every class of observations. QDA is particularly useful if there is prior knowledge that individual classes exhibit distinct covariances. A disadvantage of QDA is that it cannot be used as a dimensionality reduction technique.
Logistic regression is popular for classification when we have two classes, true or false?
True
LDA is useful when n is large, true or false?
False, it is useful when n is small
