logistic regression

Question 1

disadvantage of linear model?

Answer 1

predicted probabilities may be below 0 or above 1

Question 2

what does logic(p) equal to?

Answer 2

ln(p/1-p)=β0+β1*x (β1 is the expected increase in log-odds when X increases by one unit)

Question 3

intercept in odds?

Answer 3

e^β0

Question 4

slope

Answer 4

e^β1

Question 5

can estimate β be interpreted as a change in the probability Y=1 associate with unit change in X?

Answer 5

No. Odds not linear

Question 6

sensitivity?

Answer 6

TP/P (used if FN more costly than FP), RAISE SENSITIVITY BY CLASSIFYING MORE AS ‘YES’ (less FN but more FP, specificity reduced)

Question 7

true positive rate?

Answer 7

TP/P (Sensitivity 1 – Type 2 error)

Question 8

false positive rate?

Answer 8

FP/N (1 – Specificity Type 1 error)

Question 9

positive prediction rate?

Answer 9

TP/hat P (precision)

Question 10

negative prediction rate?

Answer 10

TN/ hat N

Question 11

what doesROC (Receiver Operator Characteristic) curve traces out?

Answer 11

true positive rate and false positive rate as we vary the probability threshold from 0 to 1

Question 12

AUC is the area under the ROC curve. what does it measure?

Answer 12

it measures overall performance of classifier (max AUC=1) the larger the better the classifier

Question 13

what is the chance line?

Answer 13

random guess can produce the classifier at a 45 degree angle. no classifier should be worse than this line. AUC=0.5

Question 14

for cross validation, what is used instead of MSEs

Answer 14

number of misclassified observations

Question 15

converting factor variable for numeric linear regression (has negative values so ignore)

Answer 15

Default$default_yes = ifelse(Default$default == “Yes”, 1, 0)
lm_fit = lm(default_yes ~ balance, data = Default)
summary(lm_fit)

Question 16

to tell R to use logistic regression,

Answer 16

use family=binomial
e.g.glm_fit1 = glm(default ~ student,
data = Default, family = binomial)
summary(glm_fit1)

Question 17

to make predictions?

Answer 17

predict(glm_fit1,newdata=data.frame(variable,c(option1,option2 etc)),type=’response’)

Question 18

find probability of first 10 predictions?

Answer 18

glm_prob = predict(glm_fit2, type = “response”)

glm_prob[1:10]

Question 19

confusion matrix of probability threshold of 0.5?

Answer 19

confusion_matrix=table(glm_prob>0.5,Default$default)

Question 20

to find AUC(area under ROC curve)?

Answer 20

pred=prediction(glm_prob,Default$default)
perf=performance(pred, measure=’tpr’, x.measure=’fpr’)
auc_perf=performance(perf, measure=’auc’)
round(auc_perf@y.values[[1]],2)

Question 21

plot ROC curve with chance line?

Answer 21

plot(perf)
abline(0,1,lwd=1,lty=2)
# Add text to the ROC plot
text(0.4, 0.8, paste(“AUC =”, round(auc_perf@y.values[[1]], 2)))

Question 22

find accuracy?

Answer 22

accuracy_perf = performance(pred, measure = “acc”)
plot(accuracy_perf, col = “deeppink3”, lwd = 2)
ind = which.max(slot(accuracy_perf, “y.values”)[[1]])
acc = slot(accuracy_perf, “y.values”)[[1]][ind]
cutoff = slot(accuracy_perf, “x.values”)[[1]][ind]
print(c(accuracy = acc, cutoff = cutoff))

Question 23

add accuracy point on curve?

Answer 23

points(cutoff, acc, type = “p”)

text(0.6, 0.86, “(0.4299, 0.9740)”, cex = 0.85)

Question 24

most accurate confusion matrix?

Answer 24

confusion_matrix=table(glm_prob>cutoff, Default$default)

confusion_matrix

Question 25

find point on ROC?

Answer 25

sensitivity = 124 / (124 + 209)
specificity = 9615 / (9615 + 52)
TPR = sensitivity
FPR = 1 - specificity
plot(perf)
abline(0, 1, lwd = 1, lty = 2)
text(0.4, 0.8, paste("AUC =", round(auc_perf@y.values[[1]], 2)))
points(FPR, TPR, type = "p", pch = 16)

Question 26

model validation: validation set approach

Answer 26

set.seed(100)
#generate 5000 random numbers from 1-10000
ind=sample(10000,5000)
training = Default[ind, ]
testing = Default[-ind, ]
glm_train = glm(default ~ balance + student, 
                data = training,
                family = "binomial")
summary(glm_train)
glm_prob = predict(glm_train, 
                   newdata = testing,
                   type = "response")
glm_pred = rep("No", 5000)
# Classify the individual to the default category if the posterior probability is greater than 0.5
# That is, replace "No" with "Yes" in vector if the predicted probability in "glm_prob" is greater than 0.5
glm_pred = ifelse(glm_prob > 0.5, "Yes", "No")
# Confusion matrix
table(glm_pred, testing$default)

Question 27

another method to find accuracy?

Answer 27

accuracy = function(response, predict) {
  mean((predict <= 0.5) & response == 0 | (predict > 0.5) & response == 1)
}
#predict<=0.5 & response ==0 are the true positives
# We can verify if the cost function is written correctly
response = ifelse(testing$default == "Yes", 1, 0)
predict = glm_prob
accuracy(response, predict)

Question 28

k-fold CV

Answer 28

set.seed(100)
glm_fit2=glm(y~var1+var2+var3, data=Default, family=binomial)
cv_error = rep(0, 2)
# Store the K-fold error rate into cv_error Use K = 5 and K = 10
cv_error[1] = cv.glm(Default, glm_fit2, accuracy, K = 5)$delta[1]
cv_error[2] = cv.glm(Default, glm_fit2, accuracy, K = 10)$delta[1]
cv_error