Logistic Regression

Question 1

Cross Entropy?

Answer 1

Summation of Negative Log of Maximum Likelihood

Question 2

Why does closed form solution not exist for Logistic Regression?

Answer 2

This is because we use maximum likelihood to estimate the coefficient of the model, and since sigmoid function is non linear, it is not possible to present a closed form solution.

Question 3

What is Odds Ratio?

Answer 3

Odds = p(x) / (1 - p(x))
Likelihood that an event will occur expressed as a proportion of likelihood that the event will not occur.

Odds ratio measures the association between exposure and an outcome It measures the odds of an outcome in presence of exposure compare to odds of an outcome in absence of exposure.

OR > 1, represents increased occurrence of an event.
OR < 1, represents decreased occurrence of an event.

Odds are used in Horse Racing.

Question 4

What is Log-odds/Logit?

Answer 4

Log(Odds Ratio) = Y

Y = Bo + B1X

Question 5

Why can’t we use ordinary least square method in Logistic regression

Answer 5

Because when we differentiate the error term, we will get the simultaneous equation consisting of exponential terms and it is not possible to solve it.

Question 6

Which statistic do we use in the case of Logistic regression to check the significance of the regressor variable?

Answer 6

Z-Statistic

Question 7

Confounding Variable

Answer 7

A variable that potentially has an effect on the outcome of a study or experiment, but is not accounted for or eliminated.

Confounding variable is related to both predictor variable as well as the response variable.

However, In case of collinearity, the variables are related to predictor variable only.

Question 8

Derivative of σ(z)

Answer 8

σ(z)[1-σ(z)]

Question 9

Loss function for Multinomial Logistic Regression

Answer 9

L = - [y1log(y1_pred) + y2log(y2_pred) + y3log(y3_pred) + …….]

For 2 class classification problem, it reduces to:
L = - [y1log(y1_pred) + y2log(y2_pred)]; where y2 = (1-y1)