9) Quadraric Forms Orthogonal And Symmetric Matrices Flashcards
INNER PRODUCT
Properties
The inner product on R^n is the map: The dot product R^n X R^n -> R, (x,y) -> x•y =(x,y) = SUM( x_i y_i) = x^T y Sum from I=1 to n.
Properties: 1) symmetry: x•y = y•x
2) bilinearity: for all x,y,x’,y’ in R^n and for all s,t in R: (sx + tx’) •y = sx•y + tx’•y and x•(sy+ty’) = sx •y + tx •y’
3) non negativity: x•x is always bigger than or equal to 0
Length of vector x
|x| = SQRT( x•x) as always bigger than or equal to 0
x=0 if and only if the length of x is 0
So if length is 0 it’s the 0 vector
If x is not the 0 vector then x/ |x| has length 1 (UNIT VECTOR)
Orthogonality for vectors if x,y in R^n are orthogonal
x,y in R^n are orthogonal if x•y =0
E.g. If inner product =0
If x^T y =0
Orthonormal
A set if vectors v_1,v_2,…,v_n in R^n is called orthonormal if:
v_i • v_j = kronecker delta function = { 1 if I=j, 0 if i≉j}
Eh same vectors dot to give 1 -> length is 1
All others dot gives 0 -> orthogonal
Orthogonal set all with length 1
Theorem for mutually orthogonal vectors
ANY SET OF: n mutually orthogonal Non-zero vectors v_1,...v_n in R^n Form a BASIS FOR R^n.
As they are linearly independent: shown by dot of each to give scalars must be 0. I.e. Taking inner product gives kronecker delta function.
Orthogonal Matrices definition
A nXn matrix is orthogonal if A subset of M_n(R) (Real square Matrices):
Ax•Ay = x•y for all x and y in R^n.
(Ax)^T Ay = x^T A^T Ay =x^Ty
Hence ANY ORTHOGONAL matrix A has an inverse A-1 = A^T
A ∈O(n) if and only if it’s columns, as a set of vectors form an orthonormal set
Orthogonal Matrices properties
The identity matrix E is an orthogonal matrix.
•if A and B are orthogonal Matrices then so is AB
- if A is an orthogonal matrix so is A^T = A^-1
- if A is an orthogonal matrix, then |A| + +1 or -1
Converse not true: there are plenty of Matrices with det A plus or minus 1 that aren’t orthogonal but given that it is orthogonal we know the determinant is either 1 or -1
Orthogonal Matrices
• As a set
The set of all orthogonal nXn Matrices is a group O(n) = {A subset M_n(R) | A^T A =E}
= {A in M_n(R) | A^-1 = A^T }
• A ∈O(n) if and only if it’s columns, as a set of vectors form an orthonormal set by seeing that elements in matrix A^TA are identity matrix elements relating to kronecker delta function
Examples of orthogonal Matrices
Orthogonal Matrices det 1 or -1 but converse not true.
Eh rotation matrix = (cosx, -sinx, sino, cosx)
And permutation Matrices
Symmetric Matrices
A matrix A⊂M_n(R) or a linear map A: R^n -> R^n, x-> Ax
Is called symmetric if
Ax•y = x•Ay for all x,y in R^n
A MATRIX A IS SYMMETRIC IF AND ONLY IF A^T = A
Shown as applying symmetry to inner product and equating
Theorem for symmetric matrix and eigenvalues and eigenvectors
Let A⊂M_n(R) be a SYMMETRIC MATRIX. Then:
1) all eigenvalues of A are REAL numbers
2) for vectors u and v. Au = tu and Av =sv and s≉t then
u•v =0
I.e. For eigenvectors of A with DISTINCT eigenvalues then they are orthogonal with inner product 0
Symmetric matrix and basis
If A is an nXn symmetric matrix then there is an ORTHONORMAL basis for R^n consisting of eigenvectors for A.
BASIS FOR R^n
If NORMALISED THEY DO!!!
As if for n distinct eigenvalues for each eigenvector corresponding length one can be found. Then we have that the set is orthonormal as every dot gives required kronecker delta function.
Since characteristic equation for A is polynomial degree n even eigenvectors form a basis for R^n when eigenvalues not all distinct.
Corollary of symmetric matrices and diagonal
If A is a symmetric nXn matrix then there is an orthogonal matrix P in O(n) and nXn diagonal matrix D st
A = PDP^-1 = PDP^T
Of NORMALISED EIGENVECTORS
For coordinates on V the orthonormal basis reduces to dimensions
Quadratic forms
A quadratic form on R^n is a function of the form
Q(x) = ΣΣq_ij x_i y_j sums from i=1, j=1 to ∞
Set of n^2 constants q_ij in R.
Since we have x_ij = x_ji this implies that b_ij ={ q_ij if I=j, or mean of q_ij and q_ji if i not equal to j)
That is in matrix form coefficients of x_ix_j and x_jx_i are equal.
Symmetrically quadratic form is Q(x) = ΣΣb_ij x_i y_j Where b_ij = b_ji
Defined as a matrix:
Q(x) = x^T B x = (x_1,…,x_n) B Col_vector( x_1,…,x_n)
B is symmetric matrix due to coefficients b_ij.
Example of quadratic form
• Q(x) = x_1 ^2 + 4 x_1 x_2 -3x_2 ^2 is a quadratic form in R^2 with coefficients: q_11= 1, q_12= 4, q_21=0, q_22= -3. Written in symmetric form:
B=R1 [1,2]
R2[2,-3]
Halving the sum of q_12 and q_21
Deriving Canonical form
Since B is symmetric it has those properties:
B=PDP^T where P is an orthogonal matrix of columns of eigenvectors
And D us diagonal of eigenvalues.
Hence we can diagonalisa the quadratic form Q(x) =x^T B x = x^T PDP^T x = ~x^T D ~x
Where ~x = P^Tx is a vector ~x = (u_1•x, u_2•x,…, u_n•x) columns vector
Hence Q(x) = sum of eigenvalues * ~x ^2 for each ~x_i up to n. You
Canonical form
The quadratic form Q(x)?= x^T B x =
( y_1 ^2 +… + y_r ^2) - ( y_(r+1) ^2 +…+ y_(r+s)^2)
Where y_i = v_i• x, are linear combinations of x_i.
Eigenvalues λ_i are ordered:
The first r are positive, the next s are negative and the rest 0 ( In total all n)
v_i =
• sqrt( λ_i) u_i if i is less than or equal to i.e. For λ_i positive
• sqrt( -λ_i) u_i if I is bigger than or equal to r+1 and less than or equal to r+s
I.e. For λ_i is negative
• u_i if i in [r+s,n] those λ_i=0
Rank and signature of quadratic forms
The rank of Q is r+s and signature is r-s
For canonical form
That is rank is the number of positive and negative eigenvalues, non zero
r-s is the difference between the two with number of positive taken negative (how much more positive)
a quadratic form is positive definite, negative definite, semi definitive def
Positive definite: positive for all x not equal to 0.
Negative definite/ negative for all x not equal to 0.
Positive semidefinite: Q(x) is bigger than or equal to 0 for all x not equal to 0
Negative semidefinite: Q(x) is less than or equal to 0 for all x not equal to 0
INDEFINITE IF Q(x) can take both positive and negative values
HENCE WE CAN DETERMINE THE NATURE OF THE QUADRATIC FORM by determining eigenvalues of symmetric matrix B
Distance squared function has rank n and signature n
a quadratic form is positive definite, negative definite, semi definitive relation with eigenvalues
Positive definite if all positive eigenvalues
Negative definite if all negative eigenvalues
Becomes semidefinite if includes some =0
Indefinite if at least one positive one negative eigenvalues
Example determine the canonical form of the quadratic form
Q(x) = 5x_1^2 + 6x_2^2 + 7x_3^2 -4x_1x_2 + 4x_2x_3
Where x = (x_1, x_2,x_3)^T what is the rank and signature
We can write as
Q(x) = x^T B x
Where B is the symmetric matrix
[5 , -2, 0]
[-2, 6, 2]
[0, 2, 7]
The corresponding eigenvalues and eigenvectors can be normalised.we check these are an orthonormal set.
3 and (1/3) (2,2,-1) 6 and (1/3)(2,-1,2) 9 and (1/3)(-1,2,2) The canonical form is therefore Q(x) =y_1^2 + y_2^2 + y_3^2 Where y_i =v_i •x and hence y_1 = (√3/3)(2x_1 +2x_2-x_3) y_2 = (√6/3)(2x_1 -x_2+2x_3) y_3 = (3/3)(-x_1 +2x_2+2x_3)
Q(x) is therefore positive definite with rank 3 and signature 3
Hessian matrixDeriving
Determine the nature of stationary points of a function of more than two variables
If x in R^2 and if f(x_1, x_2) has continuous first and second derivatives with stationary point at (a,b). Then the Taylor series …
f(a+h, b+k) = f(a,b) + 0.5 ( f_11 h^2 + 2f_12 hk + f_22 k^2) +…
In matrix form
f(a+h, b+k) = f(a,b) + 0.5 (h k) H col_vector(h,k)
H = hessian = [f_11, f_12]
[f_21, f_22]
Symmetric as second partial derivatives same
(In addition stationary points able to determine by eigenvalues)
Stationary points and hessian
For f(x), x∈R^n that has a stationary point at x=x_0
f(x_0 + δx) = f(x_0) +0.5(δx)^T H (δx) +…
= f(x_0) + 0.5 Q(δx) +…
Where Q(δx) is a quadratic form associated with the symmetric matrix H whose components ate partial second derivatives of f evaluated at x=x_0
h_ij = (∂^2 f/ ∂x_i ∂x_j. ) (x_0) for I,j = 1,…,n
The stationary point is:
A LOCAL MINIMUM if the quadratic form Q(δx) is positive definite.
A LOCAL MAXIMUM if the quadratic form Q(δx) is negative definite.
Q(δx) is indeterminate then stationary point is a saddle
If Q(δx) is either positive or negative semidefinite then further investigation in order to to determine its nature
Stationary points and hessian eigenvalues
If a function f(x) for x in R^n that has CONTINUOUS FIRST AND SECOND derivatives has a stationary point at x=x_0 and if the HESSIAN H at x_0 has eigenvalues λ_1,…, λ_n
- x_0 is a local minimum if λ_i is bigger than 0 for I=1,…,n
- x_0 is a local maximum if λ_i is less than 0 for I =1,…,n
- x_0 is saddle point if at least one eigenvalues is positive and at least one negative AND NONE 0
• if any at all λ_i =0 then we require further investigation
Example find and determine nature of stationary points of the function f(x,y) = x^2 -4xy + 3y^2 -2x +y -2
Finding partial derivatives for the hessian: f_xx =2 f_xy=-4 f_yy=6 f_yx=-4
Hessian is [2,-4]
[-4, 6]
Has characteristic equation giving eigenvalues which have DIFFERENT SIGNS and thus the stationary point is a saddle point
Example find and determine nature of stationary points of the function f(x,y,z) = 2x^2 -2xy + 3y^2 + z^2 +4xz
Has stationary point at f_x = f_y = f_z =0
f_x = 4x -2y + 4z
f_y = 6y - 2x
f_z =2z +4x
Hence stationary point only at (0,0,0)
Has hessian matrix [4,-2,4]
[-2,6,0]
[4,0,2]
Looking at characteristic equation product gives a negative value therefore they are either all negative or only one is negative.
Sum is a positive value and hence they can’t all be negative. Hence we have 2 positive and one negative eigenvalue. It has rank 3 and signature 1 and the point is a saddle point
