Week 3 Flashcards
What are the four fundamental subspaces?
- Column Space C(A)
- Null Space N(A)
3.Row Space R(A)
4.
What is column space?
Column Space C(A) is the span of columns of a matrix (u1,….,un) - all linear combination of vectors {u1,u2…,un}
where is column space used?
Used in solving equations of the form Ax=b. i.e., for what b does Ax=b have solution? Ans: for all b belongs to C(A)
What is a null space?
N(A) = {x | Ax=0}
What are the two conditions for subspace?
1) if x1 and x2 belongs to Null Space of A then x1+x2 also should be in null space of A
- If x belongs to N(A) and alpha is a scalar then A(alpha.x)=alpha.A.x =0
So, alpha.x belongs to N(A)
how to find an x such that Ax=0
It is synonymous to ask for a linear combination of the columns of A should result in the zero vector.
If A is invertible, then what can we say about the null space of A i.e., N(A)?
N(A) has “Zero” only and C(A) is the whole space.
How to find Null Space of a matrix A?
we can use Guassian elimination to find the Null space of matrix A
What is Rank and Nullity?
Rank is the number of Pivot Columns and Nullity is the number of free variables.
Rank = dim(C(A)
Nullity = dim (N(A))
If A has n columns ? then Rank + Nullity = n (Rank Nullity theorem)
What is Row Space?
Row Space is column space of AT (A Transposed) = span of rows of A.
R(A) = C(A Transpose)
What is Column Rank ?
Column Rank = dim (C(A))
What is row rank?
row-rank = dim(R(A))
What is the left Null SPace?
Left Null Space N(A-Transpose) = {y | A-Transpose.y =0 }
= {y| y-Transpose .A =0}
For a mxn matrix A,
[y1… ym] [A] = [0….0]
it is the linear combination of rows leading to zero vector.
how are row space and left null space related?
dim (C(A-Transpose) + dim(N(A-Transpose)) = Number of rows =m
we know dim(C(A) =r
therefore r+ dim(N(A-Transpose)) = m
=> dim(N(A-Transpose)) =m-r
what is length of a vector?
Length of vector is the distance of the vector from origin. alias norm of the vector which is square root of squares of the coordinates of the vector.
What are orthogonal vectors?
Two vectors are orthogonal when their inner product is zero.
What is inner product?
if x and y are 2 vectors then x-Transpose.y=0 i.e., coordinate wise multiplication
Which vector is orthogonal to all vectors?
zero vector is orthogonal to every vector x
What is the relation between orthogonality and linear independence?
If {v1,v2,….,vk} are mutually orthogonal “non-trivial” set of vectors, then {v1,…vk} is a linearly independent set.
On the contrary, if 2 vectors are linearly dependent, then they are not orthogonal (perpendicular) (rather parallel.)
What are orthonormal vectors?
vectors {u,v} is orthonormal if u-Transpose.v =0 and ||u|| and ||v||=1
What are orthogonal subspaces?
U, V are orthogonal subspaces if x-Transpose.y=0, for all x belongs to U and y belongs to V.
What are the orthogonality relations w.r.t. four fundamental subspaces of a matrix A?
- R(A) is orthogonal to N(A)
Note : If Ax=0 where A is mxn matrix then it means that row 1 of matrix A is orthogonal to x and row 2 is orthogoanl to x and all the way to row m orthogonal to x. Not only rows but any linear combination of the rows are also orthogonal to x. therefore R(A) is also orthogonal to N(A) - C( A-Transpose) is orthogonal to N(A)
- C(A) is orthogonal to N(A-Transpose)
What are projections?
finding the shadow of one vector on the other normally incident at 90 degrees.
Remark: we want to project vector b onto vector x which will be incident at point p on vector x.
Why do we need projection?What are inconsistent system?
A system of equations which does not have any solution is called “Inconsistent”
In terms of Matrix, Ax=b, Inconsistent systems means b does not belong to C(A). In such situations, it makes sense to project b onto C(A)
What is project on to a line?
b is a vector which needs to be projected onto a at point P which is perpendicular to x. the length of the projection c=b-p
c is perpendicular to a
project p = x-hat .a
c=b-p=b-x-hat.a
(b-x-hat.a) perpendicular to a
a(b-x-hat.a)=0 leading to x-hat = a-Transpose b divided by a-Transpose.a
p= x-hat.a= ((a-Transpose / a-Transpose.a).a
what are cauchy -Schqarz inequality?
|a-Transpose.b| <= ||a|| . ||b||
What is a projection matrix?
P= a.a-Transpose / a-Transpose.a .
then, projection of b onto a is Pb.
To project any vector b, just left multiply by Proj matrix P.
Observe: P is symmetric
P^2 =P ie., P^2b = Pb (Idea: Pb is already on the line through a. So, another round of projection wont change it)
what is least squares method? where is it used?
Taking derivative and finding the minima turns out to be the same as performing a projection
