03 Random Variables: Covariance, Correlations & Linear Model

Question 1

What is covariance?

Answer 1

• measure of the joint variability of two random variables

- strength of dependence of variables

Question 2

What is the difference of variance and covariance?

Answer 2

variance = covariance with itself

Question 3

What is the covariance between x ∈ ℝ¹ and y ∈ ℝ¹ ?

Answer 3

σₓᵧ = E[xy] - E[x] E[y] = Cov (x, y)

Question 4

What does a Cov(x,y) > 0 indicate?

Answer 4

that there is a positive relationship between x and y

Question 5

What is the variance of x ∈ ℝ¹ ?

Answer 5

σₓₓ = Var(x) = E[x²] - (E[x])²

Question 6

What is the covariance matrix of x ∈ ℝᵖ ?

Answer 6

Σₚₓₚ = E[(x - μ)(x - μ)ᵀ] => Matrix of ( σₓ₁ₓ₁…. etc) = Matrix of (σ₁₁ … etc)

Question 7

How does the covariance help to describe the properties of X ∈ ℝᵖ ?

Answer 7

X ~ (μ, Σ) = X ~ ( E[x], Var [x] )

Question 8

What is Σₓᵧᵀ?

Answer 8

Σᵧₓ

Question 9

What is Σₓᵧ ?

Answer 9

Cov (X, Y) = E[(x - μ)(y - E[y])ᵀ] = E[XYᵀ] - E[X]E[Yᵀ]

=> Matrix of ( σₓ₁ᵧ₁…. etc)

Question 10

How can you rewrite Var(A , x+b)?

Answer 10

Var(A , x+b) = A Var(X) Aᵀ

Question 11

How can you rewrite Cov(X+Y , Z)?

Answer 11

Cov(X+Y , Z) = Cov ( X, Z) + Cov (Y, Z)

Question 12

How can you rewrite Cov(AX, BY)?

Answer 12

Cov(AX, BY) = A Cov(X, Y) Bᵀ

Question 13

What is Cov (X, Y) if X and Y are independent?

Answer 13

Cov (X, Y) = 0 ₚₓᵩ

Question 14

How do you estimate μ?

Answer 14

^μ = n⁻¹ Xᵀ 1ₙ

Question 15

What values can ρ take and what is it?

Answer 15

[-1, 1], correlation coefficient

Question 16

How can you write the correlation matrix and what do its components mean?

Answer 16

P = D⁻¹/² Σ D⁻¹/²
Σ…..Cov (X, Y)
D…..diagonal matrix of variances

Question 17

What is ρ (x, y) when x and y ∈ ℝ¹ ?

Answer 17

σₓᵧ / √ [ σₓₓσᵧᵧ ]

Question 18

What does ordinary least squares stand for?

Answer 18

That you minimze the sum of squared errors

Question 19

What are the linear regression model and two defining properties?

Answer 19

• Y = Xβ + ε
• E[ε] = 0ₙ
• Var(ε) = σ² Iₙ

Question 20

What is the estimator for β?

Answer 20

^β = ( XᵀX )⁻¹ XᵀY

Question 21

What is the estimated error term?

Answer 21

^ε = Y - ^Y

Question 22

What is the estimated variance?

Answer 22

^σ = √ [ (^εᵀ ^ε)/(n-p) ]

Question 23

How can you find D from Σ (in order to find the correlation matrtix)

Answer 23

D are the diagonal elements of Σ

Question 24

When given Y and X and you are asked to conduct linear regression analysis, what do you do?

Answer 24

1) find estimator ^β
2) calculate the predicted ^Y
3) calculate the error term ^ε
4) calculate the variance ^σ

Question 25

What can help when calculating Σ and P?

Answer 25

They are symmetric so only need to calculate half