Logistic Regression

Question 1

General characteristics Logistic regression

Answer 1

Outcome must be between 0 and 1.
Sigmoid logistic function used to model dependent variable.
Coefficients estimated through MLE.

Not possible to interpret the coefficients directly because it is on an exponential basis.

logit = logarithm of the odds, the log-odds, coefficients are the change in the log-odds.

Question 2

Odds Ratio

Answer 2

Divide the odds of the first group by the odds of the second group.

Odds of being interested in the product if the client is active, over the Odds of being interested in the product if the client is NOT active.

Question 3

Regularisation

Answer 3

Overfitting, when data has high dimensions and is sparse.

Regularisation penalises large values of the estimates. If weights are large, a small change in feature can lead to a large change in the prediction, so we use the penalised log-likelihood function.

Question 4

L1 regularisation

Answer 4

• Lasso Regression, Linear. Can also be used to do feature selection (automatically select important predictors in the model) by forcing some coefficients to be zero.

Question 5

L2 regularisation

Answer 5

• Ridge Regression. More “soft”, no feature selection. If you think all predictions are important, use this one.

Question 6

Setting a threshold

Answer 6

Quantify the costs of the misclassifications (FP and FN), this way you can allocate a budget for the misclassification of the model and use ROC curve.

Question 7

Techniques for dealing with unbalanced data

Answer 7

Oversampling: SMOTE
Undersampling.
Weighted logistic regression.