Week 2: Logistic Regression (only) Flashcards
What are two problems with the classes in log reg that can make other criteria than the (overall) misclassification rate the biggest priority?
1) An asymmeitric problem - where it is more important to correctly predict some classes than other,
2) An imbalanced problem - where classes occur with very different frequencies.
Give an example of when it can be more important not to falsely predict the negative class than to falsely predict the positive class?
In a situation where we predict health status of patiens. Falselsy predicting the negative class could lead to predicting a sick patient as healthy.
Give an example of when we can encounter an imbalanced problem?
If we are modeling a very rare disease.
What is the equation that needs to be solved to compute the decision boundary in log reg? Why?
g(x) = 1-g(x).
This since the solution are points in input space for which the two classes are predicted to be equally probable and therefore lie on the decision boundary.
What is a linear classifier?
A model whose decision boundary is linear.
What is the softmax function?
How do we write the loss function L(y.hat, y.i) more specific (for log reg)?
L[p(y.i = 1 | x.i ; theta), y.i]
Write the full cost function where the log lik. function has been simplified, and identify the loss and cost . What do we call this loss function?
xx
The loss function is called the binary cross-entropy loss.
Write only the (general) binary cross-entropy loss and simplify. Speficy this general loss for logistic regression.
What is the logistic loss?
2 ln [1+exp(y.i theta^T x.i)]
What is the log of (1/n)?
- log (n)
What is a classification rule?
A rule that maps predicted probabilities into class prediction.
What is computationally good with the likelihood function for a Bernoulli distribution used in logistic regression?
That the pdf will reduce to only one part when the probability of y = 1, and only the other part when the probability of y = (-1).
