L8 - The Multivariate Regression Model

Question 1

How do you work out the residual sum of squares?

Answer 1

• Same as a bivariate model

Min RSS = Σ_t=1^N(u_i)²

So for example less take a 3 variable model of:

Y_i= β₁+β₂X_i2+β₃X_i3+u_i

Min RSS = Σ_t=1^N(Y_i- β₁(hat) -β₂(hat)X_i2 - β₃(hat)X_i3)²

Question 2

How do we work out the RSS of a 3 variable regression model?

Answer 2

• By differentiating
• FOC conditions for this problem are:

dRSS/d(β1(hat)) = -2Σ(Y_i- β₁(hat) -β₂(hat)X_i2 - β₃(hat)X_i3)=0

dRSS/d(β2(hat)) = -2ΣX_i2*(Y_i- β₁(hat) -β₂(hat)X_i2 -β₃(hat)X_i3)=0

dRSS/d(β3(hat)) =-2ΣX_i3*(Y_i- β₁(hat) -β₂(hat)X_i2 - β₃(hat)X_i3)=0

This gives us a system of three equations in three unknownvariables. More generally, if:

Y_i- β₁+Σ_j=2^k(β_jX_ij)+u_i

then the first order conditions will yield a system of k equationsin k unknown variables.

Question 3

What are the Least-Square normal equations of the three variable regression model?

Answer 3

Y_i= β₁+β₂X_i2+β₃X_i3+u_i

has least-squares normal equations of the form:

β₁(hat)N +β₂(hat)ΣX_i2 + β₃(hat)ΣX_i3=ΣY_i

β₁(hat)ΣX_i2 +β₂(hat)ΣX_i2² + β₃(hat)ΣX_i2X_i3=ΣX_i2Y_i

β₁(hat)ΣX_i3 +β₂(hat)ΣX_i3X_i2 + β₃(hat)ΣX_i3²=ΣX_i3Y_i

This is a system of three equations in three unknowns. Solving these yields the OLS estimates.

Question 4

What happens when you divide through the OLS estimator by N?

Answer 4

β₁(hat)N +β₂(hat)ΣX_i2 +β₃(hat)ΣX_i3=ΣY_i

β₁(hat) +β₂(hat)(ΣX_i2/N)+β₃(hat)(ΣX_i3/N)=(ΣY_i/N)

β₁(hat) +β₂(hat)X_i2(bar) +β₃(hat)X_i3(bar)=Y_i(bar)

• this is the regression line that passes through the mean of the data just like a bivariate regression model

Question 5

What is the general equation to solve multivariate OLS estimators?

Answer 5

In general, let X be a matrix whose columns contain the data for the explanatory variables and let y be a vector containing the data for the dependent variable. The multivariate OLS estimator can be calculated as:

β(hat)=(X(Ubar)^TX(Ubar)^-1*X*(Ubar)^Ty(Ubar)

This is k x 1 vector where k is the number of explanatory variables (including the intercept)

In order to calculate the OLS estimator in this way we must assume:

1. k < N, where N is the number of observations
2. The X variables are linearly independent.

Question 6

What is a example of solving for an OLS estimator of a multivariate regression model?

Answer 6

variable(Ubar) = vector

• the X variable is derived from –> X(Ubar)^TX(Ubar) = X(Ubar)^-1
• X(Ubar)^TY(Ubar) = the y matrix

Question 7

When it possible to solve multivariate regression models with matrices?

Answer 7

only if X(Ubar)^TX(Ubar) is invertible

This occurs when:

1. The number of colums is < than the number of rows (K < N) where N is the number of observation and K is number of parameters
2. Columns of X(Ubar) are linearly independent (there cant be perfect correlation between any of the X variables)

Question 8

What does R-squared measure?

Answer 8

the fraction of the variance of the endogenous variable which is explained by the model

Question 9

How can the regression model be wrote in matrix form?

Answer 9

Note that we can also write the regression model itself in matrix form.

y(Ubar)=X(Ubar)β + u

where β is a k x 1 vector of unknown population parameters and u is an N x 1 vector of unoberserable random errors

We can substitute for y in the expression for the estimator to obtain:

β(hat)= (X(Ubar)^TX(Ubar))^-1X(Ubar)^T(X(Ubar)β + u)
= β+(X(Ubar)^TX(Ubar))^-1X(Ubar)^Tu

The OLS estimator is therefore a linear combination of the random errors and is itself therefore a random variable whose distribution will depend on the properties of the errors

Question 10

What are the Guass- Markov assumption in matrix form?

Answer 10

1. E(u_i)=0; i=1,…,N –> expect value of the error term is equal to zero for every value of i
2. E(u_iu_j)=0; i≠j –> the expected value of u_iu_j=0, the errors are independent
3. E(u_i²)= σ_u²; i= 1,…,N –> the expected value of the squared error term is a constant σ_u² –> the variance of the error
4. E(X’u)=X’E(u)=0 –> if X is not random and is fixed, we can take it out of the expectation operation

and finally:

5. the errors follow a normal distribution

These assumptions are essentially the same as for the bivariate regression model

Question 11

What does assumption 2 and 3 tell you about the variance-covariance matrix of the errors?

Answer 11

I_N –> identity matrix

Question 12

What happens to a multivariate regression model when the X variables are correlated?

Answer 12

Y_i= β₁+β₂X_i2+β₃X_i3+u_i

ρX_i2=X_i3

This means we could write:

Y_i= β₁+β₂X_i2+β₃ρX_i2+u_i

Y_i= β₁+(β₂ +ρβ₃)X_i2+u_i

The equation only contains one genuine independent variable and we therefore cannot estimate separate effects from the two variables

Question 13

What happens to a multivariate regression model when we have only partial collinearity?

Answer 13

The previous example was one in which there was perfect collinearity between two variables.

In cases like this the least-squares method will not allow us to estimate separate coefficients for the two right-hand side variables.

More generally we might have less than perfect collinearity:

ρX_i2 +ε_i=X_i3

where V(ε_i) ≠0 but is small relative to V(X_i2)

• a software package here will treat them both as two completely two different variables

If this is the case then it is possible to estimate separate effects using least-squares but the estimates may be very inaccurate.