Tutorial 3 - Model Misspecification, Model Choice, Model Diagnostics, Multicollinearity

Question 1

What are possible ways to model nonlinear effects?

Answer 1

• Log-transformation of y and/or x
• Higher-order polynomials in x (quadratic, cubic,…)
• Semi- or nonparametric regression (not covered here)
• Nonlinear regression models (not covered here)

Question 2

What is the calculation to get the returns to experience in this regression?

Answer 2

Question 3

How to calculate the gender wage gap in this case?

Answer 3

Question 4

What is multicollinearity?

Answer 4

Perfect multicollinearity: one regressor can be expressed as a perfect linear combination of one or several other regressors.

Question 5

What does multicollinearity mean mathematically?

Answer 5

This means that the N × K regressor matrix X does not have full column rank K
-> X’œX is singular (not invertible), thus OLS estimator β^ = (Xœ’X)⁻¹X’œy is not identifiable.

Question 6

What is the problem with this regression and how to solve it?

Answer 6

Perfect multicollinearity. One variable will drop out!
-> Easy to detect and solve: leave one category out as the reference category

Question 7

What are the consequences of “imperfect multicollinearity”?

Answer 7

• OLS coefficients still unbiased, but more unstable (i.e. coefficients might be very different if another sample was used).
• Standard errors also unbiased, but they can be large. Thus, regressors may be individually insignificant, even if they are jointly significant.
• Multicollinearity is not a problem if the aim is to predict y (rather than to estimate the effect of a single regressor on y).

Question 8

What is the Variance inflation factor?

Answer 8

a measure for multicollinearity

Question 9

How do you calculate the Variance inflation factor?

Answer 9

For variable xⱼ , the VIF is defined as below, where Rⱼ² denotes the R² from a regression of variable xⱼ on all other
covariates xₖ (k ≠ j ).

Question 10

What is the idea and interpretation of the variance inflation factor (VIF)?

Answer 10

• Idea: strong linear dependence between xⱼ and all other covariates results in a high Rⱼ²-> high VIFⱼ .
• This is called the variance inflation factor because the higher the dependence between xⱼ and the other covariates, the higher is Var( β^ⱼ ).
• Rule of thumb:
  
  • VIFⱼ > 10 implies serious multicollinearity,
  • VIFⱼ = 1 would mean that xⱼ has zero correlation with the other regressors

Question 11

What other way (except VIF) is there to check for multicollinearity?

Answer 11

test whether regressors are jointly significant -> F-Test for joint significance

Question 12

How to detect outliers/influential observations?

Answer 12

• Residual analysis: Plot std/stud residuals againt fitted values
• Cook’s distance as a measure of influence for observation i

Question 13

How do you calculate Cook’s distance?

Answer 13

Question 14

How do you interpret the Cook’s distance?

Answer 14

Rule of thumb: Cᵢ > 4/N -> large influence!

Question 15

What is the total sum of squares (“Total variance”)?

Answer 15

Question 16

What is the residual sum of squares (“Unexplained variance”)?

Answer 16

Question 17

What is the explained sum of squares (“Explained variance”)?

Answer 17

Question 18

How do you interpret a high R²?

Answer 18

A high R² means that much of the variance of y is explained by the regressors, i.e. the model fits the data well.

Question 19

How do you calculate R²?

Answer 19

Question 20

What is a problem with R²?

Answer 20

R² always increases if further regressors are added to the model, if even these regressors have no additional explanatory power.

Question 21

How do you calculate the adjusted R²?

Answer 21

Question 22

How do you calculate the mean squared error (MSE)?

Answer 22

Question 23

How can you prove the equation below?

Answer 23

Question 24

What is the definition of the Mean Squared Error of Prediction (MSEP) for model m?

Answer 24

expected squared deviation of true values from predicted values:

Question 25

Which three parts does the MSEP consist of?

Answer 25

Question 26

What is the Bias-Variance-Tradeoff, especially w.r.t. the MSEP?

Answer 26

• More flexible models (with more covariates, polynomials of higher order, interaction terms…) reflect the variation in the data better -> lower bias.
• But more flexible models also have a higher variance; and they also are more sensitive to variation in the data that is only random (especially if the data set is small). -> higher variance, danger of “overfitting”.

Question 27

How can the MSEP be estimated?

Answer 27

• |m| is the number of parameters in the model,
• σ² is a consistent (and, if possible, model-independent) estimator of the error variance Var(ϵ) = σ².

Question 28

For R², MSEP, AIC, BIC and GCV, which model do you choose?

Answer 28

• R²: highest value
• MSEP, AIC, BIC, GCV: lowest (absolute) value

Question 29

How do you calculate the AIC?

Answer 29

Question 30

How do you calculate the BIC?

Answer 30

Question 31

How do you calculate the GCV?

Answer 31

Question 32

How can you use a Validation Sample for model selection?

Answer 32

“split sampling”, i.e. randomly split the sample into two halfs. Estimate the model for the rst subsample. Then check the model’s performance using the other subsample.

Question 33

What are potential Variable Selection Procedures?

Answer 33

• Backward Elimination
• Forward Selection
• Stepwise Procedures

Question 34

How do you interpret the coefficient for “female”?

Answer 34

it is the difference between male and female with 0 experience, keeping all other variables constant