Multiple Linear Regression

Question 1

Write the regression equation.

Answer 1

y = b0 + b1 x + e

Question 2

Why will the estimate value of y not be perfect?

Answer 2

Because there are residuals or errors, which is what ‘e’ stands for.

Question 3

What is a regression line used for?

Answer 3

To make predictions about the value of the dependent variable based on a value of the predictor.

Question 4

What happens to R^2 when more variables are added?

Answer 4

R^2 always increases even when the new variables have no predictive power.

Question 5

How do we know if two models are nested?

Answer 5

If one model contains all the terms of the other, and at least one additional term.

Question 6

What is b(0)?

Answer 6

The value of y variable when x variable=0.

Question 7

What is b(1)?

Answer 7

This is the amount of change in variable y for one unit change of variable x.

Question 8

Write the multiple regression equation.

Answer 8

y = b(0) + b(1)x(1) + b(2)x(2)… + b(n)x(n) + e

Question 9

What is a fully specified model?

Answer 9

A model in which we have accounted for all factors that determine variation in the dependent variable (y).

Question 10

Why can’t we usually have a fully specified model?

Answer 10

We cannot measure all the factors that affect y

Question 11

What is the relationship between the t and p values?

Answer 11

As the t value increases, the p value increases

Question 12

What is the t value for significance at 0.05 level of confidence?

Answer 12

+/- 1.96

Question 13

What does rejecting the null hypothesis mean?

Answer 13

• Our relationship is not likely to have occurred by chance
• Our relationship is likely to be reflected in the population

Question 14

What do we do when we want to add a categorical variable such as sex to the model?

Answer 14

We create a variable that takes the values “0” and “1” for men and women, respectively.

Question 15

What do we do if there are multiple categories?

Answer 15

We create multiple dummy variables where one category is a reference and left out of the model.

Question 16

What does R^2 mean?

Answer 16

How much variability in the dependent variable is explained for, e.g., an R-squared value of 0.66 means that 66% of the variance in the y variable is
explained by the x variables.

Question 17

How do we interpret the p value?

Answer 17

If the p value is less than the alpha value of 0.05 then we know that our model has at least one significant independent variable.

Question 18

How does the forward stepwise selection method work?

Answer 18

• Begins with no variables and introduces variables one by one
• Add variables that increase R2 the most
• Continue this procedure until none of the remaining variables explain a significant amount of the additional variability in y

Question 19

How does the backwards stepwise selection method work?

Answer 19

• Starts will all variables in the model
• Drops variables that contribute least to R2
• Process continues until remaining variables explain a significant proportion of variability in y

Question 20

How do we compare coefficients that are measured in different units?

Answer 20

We standardise the coefficients into beta coefficients.

Question 21

How do we interpret beta coefficients?

Answer 21

As “a one standard deviation unit increase in x leads to a ___standard deviation unit increase/decrease in y.”
This way, we can compare continuous IVs to see which has the largest association.

Question 22

Why should we not standardise categorical variables?

Answer 22

Because a 0/1 dummy variable cannot be increased by one standard deviation

Question 23

What does the relaimpo package do?

Answer 23

Provides measures of relative importance for each of the predictors in the model by entering regression variables in all possible orders, and then averaging the changes in the R2.

Question 24

What does adjusted R^2 do?

Answer 24

The adjusted R2 controls for the number of variables we have included in our model, so it avoids the problem of R^2 increasing when more variables are added.

Question 25

How do we control for multicollinearity through plots?

Answer 25

• Observe the correlation between the predictors in the model by plotting them against each other.
• If correlation between any two predictors is strong, then one of them needs to be removed from the model

Question 26

What is Variance Inflation Factor (VIF)?

Answer 26

VIF larger than 5 or 10 indicates serious problems with collinearity.

Question 27

Why is multicollinearity a problem?

Answer 27

• If a predictor is strongly related to some other input, then we are simply adding redundant information to the model
• It can be difficult to separate the effects of the multicollinear predictors