Week 2

Question 1

What is the multivariate MSE?

Answer 1

Var(theta^hat) + (bias(theta hat))(bias(theta hat)’

Question 2

What is the multivariate normal distribution?

Answer 2

f(x) = 2 pi ^(-n/2) |𝝨|^(1/2) exp(-(1/2) (x-mu)’ 𝝨^-1(x-mu))

With mu = vector of mu
And Sigma = diagonal matrix of variances

Question 3

How can a (multivariate) normal distribution be rewritten to a chi squared distribution?

Answer 3

(x - μ)’Σ^-1(x-μ) or

If x is N(0, I.n) distributed and matrix A is symmetric idempotent of rank r then

x’Ax

Question 4

How can a Normal and chi squared distribution be rewritten to a students t-distribution?

Answer 4

x/sqrt(w/n) is t(n) distributed, this converges to a normal distribution

Question 5

How can two chi squared distributions be rewritten to a F-distribution?

Answer 5

(v/m)/(w/n) is F(m, n) distributed (given that v and w are independent.

Question 6

How can a Student’s t-distribution be rewritten to a F-distribution?

Answer 6

t^2 = F(1, n) distribution

Question 7

What can be said about the limit of a F-distribution?

Answer 7

If w/n -> inf then F(m, n) is (chi squared (m))/m distributed

Question 8

What is the derivation of the Least Squares estimation?

Answer 8

1. We need to choose a β s.t. the error is minimized, so we create function ɸ(β) = sum of u_i² = u’u = (y - Xβ)’(y-Xβ)
2. We want to minimize this function, so first we rewrite it to:

ɸ(β) = y’y = y’Xβ - β’X’y + β’X’Xβ

= y’y - 2y’Xβ + β’X’Xβ (because y’Xβ = β’X’y)

3. Then we differentiate and equate to 0:

derivative of ɸ(β) with respect to β = -2y’X + 2β’X’X = 0

gives

X’Xb = X’y, so b = (X’X)^-1X’y

4. So we get y^hat = Xb = X(X’X)^-1X’y

Question 9

What is the residual maker matrix, and why is it named like that?

Answer 9

1. M = I_n - X(X’X)^-1X’, this is a symmetric idempotent matrix (AA=A).
2. It’s derived from the residuals. i.e. e = y - Xb = y - X(X’X)^-1X’y = My

Question 10

What is the formula for b?

Answer 10

(X’X)^-1X’y

Question 11

What is the least absolute deviations estimator?

Answer 11

Instead of minimizing the sum on u_i², we now minimize the sum of the absolute value of u_i.

Question 12

What are the advantages and disadvantages of the least absolute deviations?

Answer 12

Advantages:

• Resistant to outliers

Disadvantages:

• Might have multiple solutions
• Not differentiable, so it’s a huge disadvantage in theoretical results

Question 13

How is R² derived?

Answer 13

1. We have that y = Xb + e = y^hat + e
2. Since X’e = 0, we have y^hat‘e = b’X’e = 0, thus y’y = y^hat‘y^hat + e’e. So we can create the ratio:

(y^hat‘y^hat)/(y^hat‘y^hat) = 1 - (e’e)/(y^hat‘y^hat)

3. If we create the symmetric idempotent matrix A (with Ae = e), then

Ay = AXb + Ae = AXB + e, so then

y’Ay = b’X’AXb + e’e (a.k.a. SST (a.k.a. TSS) = SSE + SSR)

4. Thus we can usually write R² = 1 - SSR/SST (aka. TSS) = 1 - (e’e)/(y’Ay)

Question 14

(AB)’ = … ?

Answer 14

B’A’

Question 15

(AB)^-1 = … ?

Answer 15

B^-1A^-1

Question 16

What is ɩ?

Answer 16

It’s a vector of ones.