Logistic Regression

Question 1

Probability vs Odds

Answer 1

Odds = P/(1-P)
e.g., odds are 5 to 3

Question 2

Logistic regression vs log of odds

Question 3

Define logit

Answer 3

logit = log(odds) = log(p/(1- p))
This is the response variable of logistic regression

Question 4

Why use sigmoid/softmax and not simple normalization?

Answer 4

We want logarithm of model’s output (i.e., softmax) to be suitable for gradient-based optimization
https://stats.stackexchange.com/questions/162988/why-sigmoid-function-instead-of-anything-else/318209#318209

Sigmoid
Gradient is nearly a constant (-1) when prediction is confidently wrong
Gradient is nearly 0 when prediction is confidently correct

Question 5

Probability vs Likelihood

Answer 5

Probability corresponds to finding the chance of something given a sample distribution of the data, while on the other hand, Maximum Likelihood Estimation refers to finding the best distribution of the data given a particular value of some feature or some situation in the data.

https://medium.com/swlh/probability-vs-likelihood-cdac534bf523#:~:text=Probability%20corresponds%20to%20finding%20the,some%20situation%20in%20the%20data.