Chapter 3.2 Multicollinearity/Homoscedasticity Flashcards
Explain multicollinearity.
–> no linear dependency between predictors
–> we can use the rank of the observation matrix to determine that
–> if two predictors are correlated, might as well omit one of them
What’s the observation matrix?
How do we know there’s an exact linear relationship? What does this also say?
It has the shape n x (p+1)
with (p+1) < n
if Rank (X) < p+1, X is singular–> i.e. can’t calculate a closed-form solution
… leads to issues wrt. the significance of predictors
high correlation between independent variables
A basic check of multicollinearity is to …
* … indicate problems.
− Large means greater than the correlations between predictors and response.
* It is possible that the pairwise correlations are small, and yet a …
calculate the correlation coefficient for each pair of predictor variables.
Large correlations (both positive and negative)
linear dependence exists among three or even more variables.
Alternatively, we can use the Variance Inflation Factor (VIF).
What would a VIF of 10 mean?
Because …, removing the predictor in question should not cause a substantive decrease in overall 𝑅^2.
The rule of thumb is to remove variables with VIF scores greater than 10.
VIF = (1-Rk^2)^-1
k is the dependent variable
A VIF of 10 ==> Rk = 90%
i.e. 90% of the variance in the predictor in question can be explained by other independent variables.
so much of the variance is captured elsewhere
Consequence - Non-Significance
If a variable has a non-significant 𝑡-value, then either
* ….
* the variable is related to the response, but it is not required in the regression because …
The usual remedy is to drop one or more variables from the model.
the variable is not related to the response, or
(Small 𝑡-value, small VIF, small correlation with response)
it is strongly related to a third variable that is in the regression, so we don’t need both
(Small 𝑡-value, high VIF, high correlation with response)
Homoscedasticity
When the requirement of a …, we have homoscedasticity.
* If the data is not homoscedastic, a different estimator (e.g., weighted least squares) might be better than OLS.
* We also assume residuals to be normally distributed.
constant variance is not violated
The spread of the data points
does not change much.
Explain heteroscedasticity.
-requirement of a constant variance is violated
- leads to biased error terms and p-values of significance tests
Explain Glejser test.
slide 28 (to review)
Explain White test.
slide 29 (to review)
