Week 3

Question 1

What are the assumptions of the Linear model?

Answer 1

1. Model is linear
2. The n x k matrix X is nonrandom and has rank k
3. The n x 1 vector u has mean zero and variance sigma^2 I.n
4. The n x 1 vector u follows a normal distribution
5. If we let Q.n = X’X/n then Q.n -> Q as n to inf, where Q is a finite and positive definite k x k matrix.

Question 2

What is meant by the assumption “The model is linear”?

Answer 2

It does not mean that x is linear, as x can be any function, but it can be rewritten to B f(x).

Question 3

What is meant by the assumption “The n x k matrix X is nonrandom and has rank k”?

Answer 3

This means that X’X is non-singular

Question 4

What is meant by the assumption: “The n x 1 vector u has mean zero and variance sigma^2 I.n”?

Answer 4

That the Cov(x, y) is zero for all

Question 5

How is proven that (X’X)^-1X’y is the BLUE?

Answer 5

We want to estimate theta (true param) using a linear function. So we can write theta hat = A’y

Now if we impose unbiasedness, then:
E(theta hat) = E(A’y) = A’X β = W’β

And we get the variance var(theta hat) = var(A’y) = A’ var(y)A
This is given by an assumption, so we get:
= A’(sigma^2 I.n)A = sigma^2 A’A

A’A is minimized when A = X(X’X)^-1W
Thus the OLS is the BLUE estimator.

Question 6

What is the vector w?

Answer 6

The vector which makes clear which value of beta we want to estimate

Question 7

How is shown that A’A is minimized?

Answer 7

If we set D = A - X(X’X)^-1W, and we have X’A = W iff. X’D = 0

A’A = (D’ + W’(X’X)^-1X’)(D+X(X’X)^-1W) ≥ W’(X’X)^-1W
QED

Question 8

What is the formula for the sample variance?

Answer 8

s^2 = y’By = (X beta + u)’B(X beta + u) = u’Bu + 2 beta’ X’Bu + beta’X’BX beta

Question 9

How is shown that the sample variance is unbiased?

Answer 9

E(s^2) = E(u'Bu) + 2 beta'X'BE(u) + beta'X'BX beta = sigma ^2 tr(B) + beta ' X' BX beta
As tr(B) = 1, X'BX = 0, we get = sigma^2