10 Techniques: PCA - Introduction

Question 1

What is principal component vector Z? (intuition)

Answer 1

reducing the dimensions of X by maximizing the variance of a standardized linear combination
= statistical procedure converting a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components (using orthogonal transformation)

Question 2

What is principal component vector Z? (formula)

Answer 2

ΓᵀX

with Γ being the matrix of all eigenvectors

Question 3

When is 𝛿 a standardized linear combination of X?

Answer 3

Iff ||𝛿||₂² = 𝛿ᵀ𝛿 = 1

Question 4

How can you interpret standardized linear combination 𝛿?

Answer 4

You calculate 𝛿ᵀX and check the result:
if it only one specific component of X (which would be the most important one) or whether the components are equally weighted

Question 5

How can you find E[•] and Var (•) of principal components transformations such as 𝛿ᵀX, ΓᵀX, X - μ, Γᵀ( X - μ)?

Answer 5

Use the properties from AX - b

• > E[AX-b] = Aμ - b
• > Var(AX-b) = AΣAᵀ

Question 6

How can you find the Cov(X, Y) of X and a principal components transformation Y (such as Y = Γᵀ( X - μ))?

Answer 6

• Start with Cov (X, Y) = E[ ( X - E[X] ) ( Y - E[Y] )ᵀ ]

- then simplify!

Question 7

How can you find the Cov(X₁, Y₁) of X and a principal components transformation Y (such as Y = Γᵀ( X - μ))?

Answer 7

1) Find the matrices A that transforms X into X₁ when multiplying AX
2) Same for Y
3) Use knowledge: Cov (AX, BY) = A Cov(X, Y) B