Lecture 7A: Logistic Regression

Question 1

What does logistic regression measure?

Answer 1

Measures the relationship between categorical dependent variable and one or more independent variables by estmating probabilities using logisitc function

Question 2

When to use Logistic Model

Answer 2

• Independent variables are continous
• Meets assumptions of linear regression models
• Distribution fits linear model but target class is binary (normal distribution)

Question 3

What does it mean to train the model?

Answer 3

It means finding the optimal values. Such that we get the best predictive performance, or, the best seperation of Y(1)’s and N(0)’s.

Question 4

Optimal Coefficients

Answer 4

● The optimal coefficients can be used to predict the unseen features (x values in the equation)

Question 5

Predictive Models

Answer 5

● Predictive models are “predictive” and they are expected to have “Errors”
○ Objective is to go as CLOSE as possible to the would be reality
○ Error is the gap between the prediction and the reality
○ Process: Feed the model with labeled data and modify the parameters to minimize the ERROR (the training process)

Question 6

Feature Importance of the Model Features by

Answer 6

○ Multiplying the coefficients by the Standard Deviation
○ Convert the data set to standardized data before getting the coefficients
○ Higher coefficient values indicate larger influence of corresponding features on Outcome (Target Variable)

Question 7

Linear regression is similar to logistic regression, except…

Answer 7

Logistic Regression predicts if something is true or false, instead of predicting something continous, like size…

Question 8

Instead of fitting a line, like we do in linear regression, in logistic regression, we fit…

Answer 8

fits an “S” shaped “logistic function”. The “S” curve goes from zero to one

Question 9

What is logistic regression usually used for?

Answer 9

It is usually used for classification

Question 10

Just like linear regression, logistic regression can work with what type of data?

Answer 10

Logistic regression can work with continous data (like weight and age) and discrete data (like genotype and astrological sign)

Question 11

Logistic regression does not have the same concept of a “residual” which is used in linear regression, so it can’t use least squares and it cant calculate R^2, instead it uses…

Answer 11

Maximum Likelihood

Question 12

In summary, logistic regression can be used to

Answer 12

classify samples, and it can use different types of data (like size and/or genotype) to do that classification. It can also be used to assess what variables are useful for classifying samples

Question 13

How do we find optimal values of the coefficients?

Answer 13

Cost Function, Loss Function, Error Function

Question 14

Cost Function

Answer 14

Alternate Terms - Loss Function

Question 15

Gap between “prediction” and “reality” is prediction “…”

Answer 15

“Error”

Question 16

In Logistic Regression, how do we minimize error?

Answer 16

WE feed the model with “labelled data” and continually modify its parameters (or coefficients) to minimize the ERROR (the training process).

Question 17

How do we know when we reached the “trained state” where the ERROR is minimal?

Answer 17

Its a mathemtical optimization problem. Various approaches.
In linear regression, we try to minimize the Mean Square Error(MSE).

Instead of using error values directly, we develop a function that will measure the “cost” or “loss” related to the error and the function is “continous”.

Make the function to be “convex” so there is a clear “global minima”