generalised inverse and the SVD

Question 1

generalised inverse:

Answer 1

the generalised inverse of a matrix A is any matrix X satisfying AXA=A - when A is nonsingular X=A^-1

Question 2

solvability of linear systems:

Answer 2

if X is any matrix satisfying AXA=A then Ax=b has a solution iff AXb=b, in which case the general solution is x=Xb+(I-XA)y, where y is an arbitrary vector

Question 3

moore-penrose generalised inverse:

Answer 3

the unique matrix satisfying the following conditions - AXA=A, XAX=X, AX=(AX), and XA=(XA), aka the pseudoinverse, denoted by A^+

Question 4

svd:

Answer 4

singular value decomposition, a matrix A has an svd A=UΣV*, where U and V are square unitary matrices and Σ=diag(σ1,…,σp), p=min(m,n) (the mxm and nxn of U and V) where σ1>=σ2>=…>=σp>=0 - if A is real, U and V can be taken real orthogonal

Question 5

singular values:

Answer 5

the σi in the svd, the positive square roots of the nonzero eigenvalues of AA* or A*A

Question 6

left and right singular vectors:

Answer 6

the columns of U and V respectively, the left singular vectors are eigenvectors of AA* and the right are eigenvectors of A*A

Question 7

existence of the moore-penrose inverse:

Answer 7

if A=UΣVinC^(mxn) is an svd then A^+=VΣ^(+)U where Σ^+=diag((σ1)^-1,…,(σr)^-1,0,…0) is nxm and r=rank(A)

Question 8

low rank approximation:

Answer 8

let the svd of A be given, write U=[u1,…,um] and V=[v1,…,vn]. if k<r=rank(A) and Ak=(k)Σ(i=1)σiuivi* then with rank(B)=k, min||A-B||2=||A-Ak||2=σ(k+1)

Question 9

projector:

Answer 9

Ps is the projector onto s if range(Ps)=s and (Ps)^2=Ps, not unique

Question 10

orthogonal projector:

Answer 10

Ps is the orthogonal projector onto s if range(Ps)=S, (Ps)^2=Ps, and (Ps)*=Ps. this is unique. also, P(s⊥)=I-Ps is the orthogonal projector onto the orthogonal complement of s denoted by s⊥

Question 11

minimum 2-norm solution:

Answer 11

for given A and b with b in range(A), the vector x=A^(+)b is the solution of minimum 2-norm amongst all the solutions to Ax=b

Question 12

u*v=0 =>:

Answer 12

(||u+v||2)^2=(||u||2)^(2)+(||v||2)^2

Question 13

least squares solution:

Answer 13

for given A and b, the vectors x=A^(+)b+(I-A^(+)A)*y where y is arbitrary, minimise ||Ax-b||2. moreover, x(LS)=A^(+)b is the least squares solution of minimum 2-norm