15. Binary Logistic Model and Logistic Regression

Question 1

What is logistic regression?

Answer 1

It is a statistical analysis method to predict the binary outcome. It predicts a dependent variable by analysing the relationship between one or more independent variables.

Question 2

What is a binary outcome variable?

Answer 2

When a response (y) is binary coded (e.g. yes or no)

Predictors can be continuous or categorical

Question 3

What is binary logistic regression?

Answer 3

Binomial Logistic Regression is the statistical fitting of an s-curve logistic or logit function to a dataset in order to calculate the probability of the occurrence of a specific event, or Value to Predict, based on the values of a set of independent variables.

Question 4

Why do we not use linear regression with a binary outcome variable?

Answer 4

Distributions of a residual would be bimodal

Variation of residuals would not be constant

Relation of X and Y is not linear

Probabilities wouldn’t be contained between 0 & 1

Question 5

How does logistic regression solve the issues that occur in linear regression of a binary outcome?

Answer 5

The goal of logistic regression is to output values between 0 and 1, which can be interpreted as the probabilities of each example belonging to a particular class.

Question 6

What function in r would be used to demonstrate binary logistic regression?

Answer 6

glm(y ~ x1 + x2, data = data, family = binomial)

Question 7

What does the logistic regression model predict?

Answer 7

Predicts the probability that y = 1 (as y can either = 0 or 1)

Question 8

What is the logistic regression model equation?

Answer 8

P(yi) = 1/ 1 + e - (B0 + B1xi1)

E = Exponential
B0 = Intercept
B1 = Capturing effect of x1 on outcome y

Question 9

Why are odds and log odds important in logistic regression?

Answer 9

Log odds convert the Logistic Regression which is a probability-based model to a Likelihood–based model so it allows the estimation of model coefficients

Log Odds are equivalent to b0 + b1 + …..

Question 10

What are odds and what is the odds equation?

Answer 10

Odds of event occurring = Ratio of the probability of event occurring to the probability of event not occurring

P(Y=1)/1-P(Y=1) can range from 0 to infinity

Question 11

What are log odds and its equation?

Answer 11

Log odds are the natural logarithm of the odds

Logits correlate to an odd and a probability (e.g. -2.21 correlates to a 0.1 chance)

Every probability can be easily converted to log odds, by finding the odds ratio and taking the logarithm.

Question 12

How can non-linear data be converted to make it linear?

Answer 12

1. Convert to probability
2. Get odds
3. Take log odds

Question 13

How are logistic regression coefficients estimated?

Answer 13

Logistic regression models are estimated using maximum likelihood estimation (MLE)

MLE finds the logistic regression coefficients that maximise the likelihood of the observed data having occurred

Larger log-likelihood values indicate poorer fitting models

Question 14

How does the method of MLE differ the the method of least squares used in linear regression?

Answer 14

While least squares estimation minimises the SSE to find the coefficients for the line of best, MLE minimises the log-likelihood

Question 15

What is the test used to evaluate our overall model in logistic regression?

Answer 15

Likelihood ratio test/ Chi-squared difference test

Compare our model to a baseline model with no predictors (null model)
Assess the improvement in fit

Question 16

What is the baseline used in a likelihood/chi-squared test?

Answer 16

Baseline model = Predicted values for DV are based on most frequent DV value (0 or 1)

• It is the best guess of DV value in absence of informative predictors
• Analogous to using the mean DV value as the baseline in linear regression

Question 17

What is used to compare to the baseline model to full model in an overall model test of logistic regression?

Answer 17

Deviance as there will be no variance (as either 0 or 1)

Deviance = -2LL (-2* log likelihood)

Question 18

How do you test overall model of logistic regression through likelihood/chi-squared test?

Answer 18

Compare the difference in deviance between baseline and full model to chi-squared distributed with df = k (no. predictors)

If there is a significant p value = Model improves baseline

Question 19

How do you interpret logistic model coefficients?

Answer 19

Beta coefficients are the change in log odds of y for every unit increase in x (holding other IVs constant)

Question 20

When the coefficient is an exponentiated coefficient, how is it interpreted?

Answer 20

The odds of y when x is equal to 0 (as it’s exponentiated)

Question 21

What is an odds ratio?

Answer 21

Logs odds aren’t always easily interpretable when DV changes so beta coefficients are converted into odds ratio which are obtained by exponentiating beta coefficients

Question 22

How do you interpret odds ratio?

Answer 22

Represents the change in odds with a unit increase in x

Odds ratio = 1 (no effect)
Odds ratio < 1 (negative effect)
Odds ratio > 1 (positive effect)

Question 23

How should deviance of residuals appear in glm() output?

Answer 23

Should be symmetric and roughly centered around 0

Question 24

What is meant by number of fisher scoring in glm() output?

Answer 24

Number of guesses it took to get the distribution (More guesses, the harder to fit)

Question 25

How do we look at the statistical significance of predictors?

Answer 25

Use Z-test (although this is prone to type II errors so supplemented by model selection procedure)

beta/ SE of beta

Z-test and associated p-value are provided in the summary output for glm()

Question 26

How do we calculate confidence intervals in logistic regression?

Answer 26

Compute confidence intervals for coefficients and associated odds ratio

If confidence interval include 1 then there is no effect

So if it doesn’t include 1 then it is significant

Question 27

How do we compare the fit of two nested logistic regression models?

Answer 27

Likelihood ratio test - provides alternative to z-test
AIC or BIC also test on nested models

Question 28

How do we compare the fit of two non-nested logistic regression models?

Answer 28

AIC or BIC