Chapter 3: Multiple Regression - Estimation Flashcards
How is multiple regression related to simple linear regression?
Multiple regression allows for the replication of a simple regression scenario by allowing for the estimation of the ceteris paribus effect of each variable in the presence of multiple variables
What’s the intuition behind the zero conditional mean assumption?
That there is no correlation between the variables within the term u and explicitly stated independent variables (Hence E(u|x)=0 or Cov(u,x)=0
When does the zero conditional mean assumption hold?
When all factors in the unobserved error term are uncorrelated with the explanatory variables
What happens when the zero conditional mean assumption does not hold?
The betas in the regression are biased, either upwardly or downwardly biased.
How can we measure the partial effect of x1 on y in the presence of multiple variables?
We can compute a regression for y on the residuals of a regression involving x1 on x2. By netting out the effects of x2 on x1, we can therefore extract the ceteris paribus effect of x1 on y since the remaining residuals are not correlated with x2.
When are the two betas (in simple linear and multiple) equal?
When the partial effect of x2, x3, x4… on y = 0
When there is no correlation between x1 and x2, x3, x4….
What is one key result that proves the unbiasedness of OLS estimates?
E(beta j hat) = beta j, meaning the average of the sample betas = the pop. value of beta.
In the presence of three independent variables (x1, x2, x3), assume that x3 is omitted and x2 is uncorrelated with x3 and x1 is uncorrelated with x3.
Given these conditions, how can be compute the expected value of beta 1 tilde (the lesser regression)?
E(beta 1 tilde) = pop. beta 1 + beta3*corr(x1, x3)
Give an equation that summarises Var(beta j)
Var(beta j)= Variance/(SSTx*(1-R^22)) where R^2 measures the explanatory power of all other variables other than x1 regarding the variability of x1.
Discuss the components of the variance formula
Error variance:
Given according to the population, and has nothing to do with samples. The larger the variance, the higher the variance of beta j, which makes sense since the more scattered the values of y, the less uniform the value of beta j/ each sample.
SST:
The more variability between the sample xj values and its sample mean, the lower the variance of beta j. This is expected since more variability is better than less variability.
R^2:
The more correlated xj and all other independent variables, the higher the variance of beta j. When R squared is 0, variance is at its smallest, which is the best case scenario since that implies that x1 is not correlated with any other independent variables (allowing for a true examination of the ceteris paribus relationship between x1 and y)
Why does multi-collinearity cause issues?
Implies that the independent variables are highly correlated (or in the case of perfect co-linearity, perfectly correlated).
This makes it difficult to measure the ceteris paribus effects of one variable on y, since an increase in x1 may also lead to an increase in x2, thereby causing an effect on y that results from a change in both x2 and x1, and not just x1.
What can lead to large variances in the value of a correlation coefficient?
A strong correlation between the variable being examined and all other independent variables.
A small sample size, resulting in a small value for SSTx.
Why does R squared increase as more explanatory variables are added to the regression?
Since r squared is SSE/SST, as the number of explanatory variables increases, SSR decreases, and SSE increases.
