Lecture 4: Covariance Matrix Estimation

Question 1

Empirical Covariance matrix

Answer 1

• p x n data
• A = [a1..an]
• Each row represents a log-return time series

C = 1/n * Sum[(ai - a_hat)*(ai-a_hat)^T]

where a_hat = average of all a (log-return time series)

Question 2

How do find an estimate of Σˆ of the true Σ based on the datapoints x1, x2..xn?

Answer 2

We should try to maximize the likelihood

L(Sigma) = Product [prob(Sigma, X)]

Changing variables (X = Sigma Inverse) and taking the log of the likelihood the problem can be written as

Max X (log det X) - Trace( CX)

Question 3

What is wrong with the empirical estimate?

Answer 3

The approach fails when the covariance matrix is not positive. When P > n (when there are more assets than observations). It does not handle the missing data, it has high sensitivity to outliers. Hence we can come up with a better estimate.

Question 4

How do we measure the estimation quality?

Answer 4

• Apply cross-validation principle.
• Remove 10% of the data
• Record new estimates
• Measure average “error” between estimates

How do we measure errors? Introduce a concept of distance between matrices, which is capture by Frobenius norm (Square root of sum of squares entries).

Question 5

Sparse Graphical Models

Answer 5

If given prices of many assets - we like to draw a graph that describes links between the prices.

Conditional independence - The pair of random xi and xj are conditionally independent if for xK fixed (k is not equal to i,j) the density can be factored:

p(x) = pi(xi)*pj(xj)

the variables xi and xj are conditionally independent iff the i, j elements of the precision matrix is zero (Sigma^-1)ij = 0

Question 6

Sparse Precision matrix estimation

Answer 6

In the maximum-likelihood estimation problem -

max X [log det X - Trace C_hat X - Lambda*Norm(X,1)]

The above provides an invertible results even if C_hat is not a positive definite, the problem is convex