Lecture 18

Question 1

Reduced SVD

Answer 1

if A is not square, k = min(n,m) singular values

Question 2

Cost SVD, LU, matmat

Answer 2

if n=m
LU 2/3n^3
matmat n^3
SVD 2n^3
else: O(mn^2 + n^3)

Question 3

PCA using SVD

Answer 3

X = U@Σ@V.T
X* = X@V

Question 4

SVD rewrite (low rank approximation)

Answer 4

A = Σi=1^n σi@ui@vi.T

ui lin indep. so Σi=1^k σi@ui@vi.T of rank k if no zero singular values (full rank)

Question 5

||U||_2?

||A||_2?

Answer 5

||U||2 = max{||x||=1} ||Ux|| = max_{||x||=1} √x.T@x = max_{||x||=1} ||x||_2 = 1

||A||2 = max{||x||=1} ||U@Σ@V.T@x|| = max_{||x||=1} ||Σx|| = σ1 = σmax

Question 6

Condition number and SVD

Answer 6

A square non-singular: cond_2(A) = ||A||.||A^{-1}|| and ||A^{-1}||=1/σmin so cond_2(A) = σmax/σmin
if rank deficient, σmin=0 and cond_2(A) = ∞

Question 7

Pseudo inverse of a diagonal matrix / of a matrix A

Answer 7

if Σ not invertible, Σ^+ = 1/σi if σi≠0 else 0.

A^+ = V@Σ^+@U.T

Question 8

Error low rank approximation

Answer 8

||A-~A||2 = ||Σi=k+1^n σi@ui@vi.T|| = σ{k+1}

Question 9

Relative error approx rank 1 :
A=UΣV.T, Σ =
.3  0  0  0
0  2.5 0 0
0   0  6  0
0   0  0 1.2

Answer 9

||A-~A1||_2 / ||A||_2

= 2.5/6

Question 10

A 77x40. How many values to store a rank 9 SVD approx? % saved?

Answer 10

9+77x9+40x9 = 1062 instead of 3080, 65% saved.