Week 3

Question 1

What are the four fundamental subspaces?

Answer 1

1. Column Space C(A)
2. Null Space N(A)
   3.Row Space R(A)
   4.

Question 2

What is column space?

Answer 2

Column Space C(A) is the span of columns of a matrix (u1,….,un) - all linear combination of vectors {u1,u2…,un}

Question 3

where is column space used?

Answer 3

Used in solving equations of the form Ax=b. i.e., for what b does Ax=b have solution? Ans: for all b belongs to C(A)

Question 4

What is a null space?

Answer 4

N(A) = {x | Ax=0}

Question 5

What are the two conditions for subspace?

Answer 5

1) if x1 and x2 belongs to Null Space of A then x1+x2 also should be in null space of A

2. If x belongs to N(A) and alpha is a scalar then A(alpha.x)=alpha.A.x =0
   So, alpha.x belongs to N(A)

Question 6

how to find an x such that Ax=0

Answer 6

It is synonymous to ask for a linear combination of the columns of A should result in the zero vector.

Question 7

If A is invertible, then what can we say about the null space of A i.e., N(A)?

Answer 7

N(A) has “Zero” only and C(A) is the whole space.

Question 8

How to find Null Space of a matrix A?

Answer 8

we can use Guassian elimination to find the Null space of matrix A

Question 9

What is Rank and Nullity?

Answer 9

Rank is the number of Pivot Columns and Nullity is the number of free variables.

Rank = dim(C(A)
Nullity = dim (N(A))

If A has n columns ? then Rank + Nullity = n (Rank Nullity theorem)

Question 10

What is Row Space?

Answer 10

Row Space is column space of AT (A Transposed) = span of rows of A.

R(A) = C(A Transpose)

Question 11

What is Column Rank ?

Answer 11

Column Rank = dim (C(A))

Question 12

What is row rank?

Answer 12

row-rank = dim(R(A))

Question 13

What is the left Null SPace?

Answer 13

Left Null Space N(A-Transpose) = {y | A-Transpose.y =0 }
= {y| y-Transpose .A =0}

For a mxn matrix A,
[y1… ym] [A] = [0….0]

it is the linear combination of rows leading to zero vector.

Question 14

how are row space and left null space related?

Answer 14

dim (C(A-Transpose) + dim(N(A-Transpose)) = Number of rows =m

we know dim(C(A) =r
therefore r+ dim(N(A-Transpose)) = m

=> dim(N(A-Transpose)) =m-r

Question 15

what is length of a vector?

Answer 15

Length of vector is the distance of the vector from origin. alias norm of the vector which is square root of squares of the coordinates of the vector.

Question 16

What are orthogonal vectors?

Answer 16

Two vectors are orthogonal when their inner product is zero.

Question 17

What is inner product?

Answer 17

if x and y are 2 vectors then x-Transpose.y=0 i.e., coordinate wise multiplication

Question 18

Which vector is orthogonal to all vectors?

Answer 18

zero vector is orthogonal to every vector x

Question 19

What is the relation between orthogonality and linear independence?

Answer 19

If {v1,v2,….,vk} are mutually orthogonal “non-trivial” set of vectors, then {v1,…vk} is a linearly independent set.

On the contrary, if 2 vectors are linearly dependent, then they are not orthogonal (perpendicular) (rather parallel.)

Question 20

What are orthonormal vectors?

Answer 20

vectors {u,v} is orthonormal if u-Transpose.v =0 and ||u|| and ||v||=1

Question 21

What are orthogonal subspaces?

Answer 21

U, V are orthogonal subspaces if x-Transpose.y=0, for all x belongs to U and y belongs to V.

Question 22

What are the orthogonality relations w.r.t. four fundamental subspaces of a matrix A?

Answer 22

1. R(A) is orthogonal to N(A)
   Note : If Ax=0 where A is mxn matrix then it means that row 1 of matrix A is orthogonal to x and row 2 is orthogoanl to x and all the way to row m orthogonal to x. Not only rows but any linear combination of the rows are also orthogonal to x. therefore R(A) is also orthogonal to N(A)
2. C( A-Transpose) is orthogonal to N(A)
3. C(A) is orthogonal to N(A-Transpose)

Question 23

What are projections?

Answer 23

finding the shadow of one vector on the other normally incident at 90 degrees.

Remark: we want to project vector b onto vector x which will be incident at point p on vector x.

Question 24

Why do we need projection?What are inconsistent system?

Answer 24

A system of equations which does not have any solution is called “Inconsistent”
In terms of Matrix, Ax=b, Inconsistent systems means b does not belong to C(A). In such situations, it makes sense to project b onto C(A)

Question 25

What is project on to a line?

Answer 25

b is a vector which needs to be projected onto a at point P which is perpendicular to x. the length of the projection c=b-p
c is perpendicular to a
project p = x-hat .a
c=b-p=b-x-hat.a
(b-x-hat.a) perpendicular to a
a(b-x-hat.a)=0 leading to x-hat = a-Transpose b divided by a-Transpose.a

p= x-hat.a= ((a-Transpose / a-Transpose.a).a

Question 26

what are cauchy -Schqarz inequality?

Answer 26

|a-Transpose.b| <= ||a|| . ||b||

Question 27

What is a projection matrix?

Answer 27

P= a.a-Transpose / a-Transpose.a .

then, projection of b onto a is Pb.

To project any vector b, just left multiply by Proj matrix P.

Observe: P is symmetric
P^2 =P ie., P^2b = Pb (Idea: Pb is already on the line through a. So, another round of projection wont change it)

Question 28

what is least squares method? where is it used?

Answer 28

Taking derivative and finding the minima turns out to be the same as performing a projection