C3

Question 1

PCA

Answer 1

Principal Component Analysis:
given n variables x1, …, xn, PCs are linear combinations x1, …, xn such that
1. they are orthogonal to each other
2. they maximize the variance of projections
3. the first PC explains most of the variance, the second PC less

Question 2

Metric Multi-Dimensional Scaling

Answer 2

Given n points p1 … pn (in highly dimensional space),
find a mapping p_i -> q_i (q in low dimensional space) that preserves the original distances as much as possible:
dist(p_i, p_j) = dist(q_i, q_j) for all i, j
Total error = sum((dist(p_i, p_j) – dist(q_i ,q_j)^2)

minimize the total error

Question 3

LLE

Answer 3

Locally Linear Embeddings:
often, interesting data lives in highly dimensional spaces, but we want to see it in low dimensions

• for each object Xi, find a few neighbouring objects
• measure distances between Xi and these neighbours
• find Yi in low dimensional space that preserves all mutual distances => optimization problem