Wk 7 - Logistic Regression

Question 1

What is the formula for linear regression? (x1, plus define components)

Answer 1

y’ = bx + c

predicted y = slope times x + constant (y intercept)

Question 2

What does b signify in regression formulas? (x3)

Answer 2

Slope
Coefficient
Amount of change in y for every unit change in x

Question 3

How is the fit of a regression line maximised/evaluated? (x2)

Answer 3

Least squares criterion:

Want minimal residuals (diff between scores and line)

Question 4

What 2 questions can we ask of any given regression model?

Answer 4

Q1: Does the predictor variable do anything useful?

Q2: Does the model provide a good fit to the data?

Question 5

What can we conclude if b = 0 in a regression model? (x1)

Answer 5

Changes in x produce no change/effect in y

Question 6

How do we assess model fit in linear regression? (x2)

Answer 6

Calculate r-square (proportion of variance accounted for)

And test for significance

Question 7

How are b and r-square related in linear regression? (x3)

Answer 7

Generally linked to some degree,
But if you move all scores similar distances from line,
b stays the same while r-square goes to hell

Question 8

What is the major limitation of linear regression method? (x2)

Answer 8

Can’t deal with categorical data

‘All or nothing’ scores, rather than continuous predictions/outcomes available

Question 9

How does what we are trying to predict change when using categorical rather than continuous DV/y variable? (x2)

Answer 9

Want to assess the change in PROBABILITY of y given b change in x
Rather than change in y scores

Question 10

What statistical method enables regression with 2 categorical outcomes? (x1)

Answer 10

Binary logistic regression

Question 11

In linear regression:
Predictors are continuous or categorical
Outcome is continuous
Predictors assumed normally distributed
Deals with linear relationships among variables

Whereas in logistic regression? (x4?

Answer 11

Predictors are continuous or categorical
Outcome is categorical
Predictors not assumed normally distributed
Deals with non-linear relationships among variables

Question 12

What are the 2 applications/questions of logistic regression? (x1, x2)

Answer 12

Predict category people belong to, given predictors
Identify predictors of particular (categorical) outcome variable
*Outcomes are exhaustive and mutually exclusive

Question 13

How does the linear regression model change for logistic regression? (x3)

Answer 13

y’ becomes a logistic function (s-shaped curve) =
1 divided by
e raised to the power of the linear equation (v)

Question 14

In logistic regression, if we substitute our largest x value for v… (x1)
And if v is very small… (x1)

Answer 14

y gets close to zero

y gets large

Question 15

What is the statistical question asked by logistic regression? (x2)

Answer 15

How many units change in x does it take

To shift the odds from favouring particular category of y?

Question 16

What are odds? (x1)

Answer 16

Expression of relative probability of an event happening vs. not happening

Question 17

How are odds calculated? (x1)

What happens to probability if you double the odds of an event occurring? (x2)

Answer 17

odds = p(event)
Divided by 1 - p(event)

Increases, but with diminishing returns

Question 18

Interpreting odds:

If odds = 1… (x1)

Answer 18

Two outcomes are equally likely

Question 19

Interpreting odds:

If odds > 1… (x1)

Answer 19

Target outcome is more probable than the alternative

Question 20

Interpreting odds:

If odds < 1… (x1)

Answer 20

Target outcome is less probable than the alternative

Question 21

What is the convenient way to compare odds of 2 events? (x2)

And what does this tell us? (x1)

Answer 21

Take their ratio
ie, divide one by the other

How many times more likely an event is for one group over an other

Question 22

Explain changes in odds related to the b (coefficient) in logistic regression? (x2)

Answer 22

In linear, b = change in outcome value

In logistic, b = change in log odds brought by unit change in x

Question 23

Explain how to interpret changes in log odds in logistic regression? (x2)

Answer 23

Have to EXPONENTIATE the coefficient

*as multiply/divide, exp/log undo each other

Question 24

Explain what Exp(b) represents in logistic regression? (x3)

Answer 24

Rather than a multiplier of x (as in linear regression)
Exp(b) is the multiplier/proportion of change on the old odds
ie, what we need to multiply old odds by to get new odds

Question 25

What is the impact of exp(b) on successive odds calculations? (x1)

Which is handy, as… (x1)

Answer 25

For each unit change in x, you get diminishing returns in change in new odds

This is what gives us the characteristic s-shape curve

Question 26

Despite technical changes from direct predictions to changes in odds, linear and logistic regression remain conceptually similar in that… (x1)

Answer 26

Coefficients reflect predictive utility of our predictor variables

Question 27

What are the 2 key questions in evaluating a logistic regression model?

Which are answered with which 2 tests?

Answer 27

Does the predictor variable(s) do anything useful?
Does the model provide a good fit to the data?

Significance tests on the coefficients (t-test against b = 0)
Evaluation of R2 via F test against R2 = 0 (model explains zero variance)

Question 28

In what 2 ways can the null hypotheses for testing b (the significance of the coefficient) be expressed in logistic regression?

Answer 28

b coefficients = 0: No change in log odds with increases in predictor
Exp b = 1: No proportionate change in odds with increases in predictor

Question 29

What 2 specific tests are used to evaluate coefficients in logistic regression? (plus explain/interpretation, x3, x3)

Answer 29

Wald test:
Form of chi-square testing b (change in log odds)
*significance means reject the null - evidence for predictive utility

95% CIs:
Interval we are 95% sure contains true value of Exp b
*If includes the value 1, evidence of no change in odds for predictor change

Question 30

What 2 tests evaluate R-square (model fit) in logistic regression?

And we should… (x1)
Because…(x1)

Answer 30

Cox & Snell
Nagelkerke

Report both
As first is conservative and second is liberal

Question 31

How do we assess model accuracy (ie, mispredictions) in logistic regression? (x2)

Answer 31

Omnibus test of model coefficients

Hosmer and Lemeshow test

Question 32

What is involved in omnibus test of model coefficients in logistic regression? (x3)

Answer 32

Chi-square test of whether all predictors combined account for any variance

Test against the H0 that R2 = 0

Significant result implies model does better than absolutely terrible!

Question 33

What is involved in the Hosmer and Lemeshow test of R-square in logistic regression? (x3)

Answer 33

Chi-square test of how closely model predicts outcome categories

Test against the H0 that predictions are perfect

Significant result implies discrepancies between model and data

Question 34

How do we assess the proportion of correct classifications by a logistic regression model? (x4)

Answer 34

Cases originally all placed in most common category
*% correctly classified reported
Then predicted classifications are compared to empirical data
*% correctly classified reported

Question 35

What 3 stages of info does SPSS output give us for logistic regression?

Answer 35

Preliminary info
Block 0 output
Block 1+ output

Question 36

What preliminary info should we double check in SPSS output for logistic regression?

Answer 36

How outcome categories have been coded

Question 37

What info is given in Block 0 output in SPSS for logistic regression? (x3)

Answer 37

All cases placed in most frequent category in data
Gives baseline model (no predictors) for comparison with more complex models
*Not theoretically interesting, but important

Question 38

What info is given in Block 1+ output in SPSS for logistic regression? (x3)

Answer 38

Summary and tests of R-square
Tests of variables in Logistic Regression equation (e.g., coefficients)
Info on classification accuracy

Question 39

What are the 3 methods of conducting logistic regression?

Answer 39

Direct or Enter method
Sequential logistic regression
Stepwise logistic regression

Question 40

Describe the Direct or Enter method of logistic regression (x3)

Answer 40

As linear MR - all predictors entered simultaneously (ie, Block 1)
Used to evaluate relative strength of predictors
Doesn’t test hypotheses about order/importance of each predictor

Question 41

Describe the Sequential logistic regression method (x3)

Answer 41

As HMR - researcher chooses order of entering predictors in separate blocks
Determines predictive value of each variable in context of whole model
First predictor explains max possible variance, more added if they improve fit

Question 42

Describe the Stepwise logistic regression method (x2)

Which is done in what 2 ways?

Answer 42

Predictors entered sequentially, included in model on statistical grounds
Often more exploratory/for hypothesis generation

Forward method - no predictors, progressively weaker predictors added
Backward - start with full model, progressively stronger predictors removed

Question 43

How is logistic model fit evaluated during stages of Stepwise regression? (x3)

Answer 43

Does adding this predictor significantly improve model fit?
Does removing this predictor significantly harm model fit?
Evaluated via nested model comparisons