Multiple Linear Regression

Question 1

How does multiple linear regression differ from simple linear regression?

Answer 1

▪️Includes more predictors/IVs
▪️See if they fit a regression plane (instead of a line)

y = β0 + β1x1 + β2x2

Question 2

When do you use multiple linear regression?

Answer 2

To study relationship between one DV and two or more IVs simultaneously

Question 3

What is the partial regression coefficient (βi)?

Answer 3

The amount Y will change for each uni increase in IV x1 whilst holding all other variables constant

Question 4

What is the null hypothesis for a multiple linear regression?

Answer 4

Holding all other variables constant, there is not linear association between y and x1

Question 5

What is a confounding variable?

Answer 5

Any variable that may distort the observed association between and explanatory variable and outcome

Has an effect on both variables

Question 6

What happens if you don’t take confounder into account?

Answer 6

Introduce bias in the estimation of β1

Question 7

How many confounder can you consider in a multiple regression model?

Answer 7

Usually one IV for each 10 observations

E.g. if n=100, can consider 10 IVs, 9 confounder

Question 8

What is R-squared?

Answer 8

The coefficient of determination

Measure of how well regression line/hyperplane approximates real data points (goodness of fit)

Question 9

What does an R-squared of 0 indicate?

Answer 9

Poor fit - regression line would be perfectly horizontal

Question 10

What does an R-squared of 1 indicate?

Answer 10

Perfect fit

Question 11

What is R-squared in a simple linear regression?

Answer 11

Pearson’s coefficient squared (r-squared)

Question 12

How do you interpret R-squared?

Answer 12

The proportion of variance in the DV that is “explained” by the IVs in the model

Question 13

How is R-squared interpreted in the context of prediction analysis?

Answer 13

How well the model will be able to predict values of Y based in observed values of IVs

Question 14

What does R-squared NOT indicate?

Answer 14

▪️The IVs CAUSE changes in the DV
▪️Correct type of regression was used
▪️Most appropriate IVs were chosen
▪️There’s enough data for a solid conclusion

Question 15

What is adjusted R-squared?

Answer 15

Modified to adjust for the number of IVs in the model.

R-squared increases whenever a new IV is added regardless of how informatige it is

Question 16

What statistic is the better indicator for model selection?

Answer 16

Adjusted R-squared

Question 17

What do we assume about the relationship of variables for multiple regression inference?

Answer 17

It’s linear

Question 18

What does a partial residual plot show?

Answer 18

The net relationship between X1 and Y where the influence of other variables is partialled out

Question 19

What do we assume about the residuals/error terms for multiple regression inference?

Answer 19

They’re approximately normally distributed

Question 20

How do we plot a partial residual plot?

Answer 20

Residuals of DV against residuals of each IV separately

Question 21

What is the residual error?

Answer 21

Variation in Y not explained by predictor

Question 22

How can we see whether the error terms are normally distributed?

Answer 22

▪️Histogram
▪️Normal P-P plot - plotted against theoretical normal distribution

Question 23

What is homoscedasticity?

Answer 23

▪️Stability in variance of residuals
▪️A scatterplot of standardised residuals and standardised predicted values shows no pattern
▪️Error values have same variance irrespective of x

Question 24

Observations used for regression models must be ___________

Answer 24

Independent

(can’t use repeated measurements, paired data or matched data)

Question 25

What assumptions do we have to check for when interpreting multiple linear regression?

Answer 25

▪️Linearity
▪️Normal distribution of residuals
▪️Homoscedasticity (ZPred vs Zresid)

Question 26

R-squared is a coefficient for what?

Answer 26

Measuring how well the regression line/hyperplane approximates the real data points