Logistic Regression

Question 1

Limits of Linear Regression

Answer 1

• Assumes response variable is normally distributed

Question 2

In logistic regression, the coefficients are estimated

using a technique called _______________

Answer 2

Maximum Likelihood Estimation (MLE)

Question 3

Unlike the _________________ method
used by linear regression, finding a closed form for
the coefficients using MLE is not possible. Instead, the process is _________.

Answer 3

Ordinary least Squares (OLS)

Iterative

Question 4

The ___________________________is an extension of
linear regression that allows for linear predictors to be
related to a response variable that is not normally
distributed by using a transformation or link function

Answer 4

Generalized Linear Model (GLM)

Question 5

The link function used for binomial logistic regression is called the _________

Answer 5

Logit / Log-odds function

log ( p / ( 1 - p) ) where p is a probability

Maps probabilities (0, 1) to (-inf, +inf)

Question 6

In Logistic Regression

For every unit increase in
tumor size, the odds of it
being malignant changes
by a multiple of ___

Answer 6

e ^ Beta where Beta is the coefficient

Question 7

If Beta<0 , the odds
that the tumor is
malignant \_\_\_\_\_\_\_ as
tumor size increases.
If Beta>0, the odds
that the tumor is
malignant \_\_\_\_\_\_\_\_ as
tumor size increases.

Answer 7

decreases

increases

Question 8

An estimate of the relative information lost by a given
model: the less information a model loses, the higher
the quality of the model

Answer 8

Akaike Information Criterion (AIC)

AIC = 2k - 2 ln ( L )
Where L is the maximum value of the likelihood function for the model
k is the # of estimated parameters of the model

Given a set of candidate models for the data, the preferred model is the one with the minimum AIC value. Thus, AIC rewards goodness of fit (as assessed by the likelihood function), but it also includes a penalty that is an increasing function of the number of estimated parameters. The penalty discourages overfitting, which is desired because increasing the number of parameters in the model almost always improves the goodness of the fit.

Question 9

Strengths of Logistic Regression?

Answer 9

-Outputs have a nice probabilistic
interpretation.
-Can be regularized to avoid
overfitting.
-Easy to implement and use.
-Very efficient to train.

Question 10

Weaknesses of Logistic Regression?

Answer 10

-Makes strong assumptions
about the data.
-Does not do well with missing
data.
-Tends to underperform when
there are multiple or non-linear
decision boundaries.
-Does not naturally capture
complex relationships.