Eigenvectors and Diagonalizable Matrices

Question 1

A diagonalizable matrix is a special type of linear operator that only corresponds to a simultaneous scaling along d different directions

Answer 1

These d different directions are referred to as eigenvectors and the d scale factors are referred to as eigenvalues

Question 2

Determinant: Geometric View

Answer 2

The determinant of a d×d matrix is the (signed) volume of the d-dimensional parallelepiped defined by its row (or column) vectors

Question 3

The determinant of an orthogonal matrix is either +1 or −1

Answer 3

One can use this result to simplify the determinant computation of a matrix with various types of decompositions containing orthogonal matrices

Question 4

Eigenvectors and Eigenvalues - A d-dimensional column vector x is said to be an eigenvector of d×d matrix A, if the following relationship is satisfied for some scalar λ

Answer 4

The scalar λ is referred to as its eigenvalue.

Question 5

The determinant of a diagonalizable matrix is equal to the product of its eigenvalues

Answer 5

Question 6

Eigendecomposition in matrix form

Answer 6

• V is an invertible d × d matrix containing linearly independent eigenvectors
• Δ is a d × d diagonal matrix, whose diagonal elements contain the eigenvalues of A
• The matrix V is also referred to as a basis change matrix, because it tells us that the linear transformation A is a diagonal matrix Δ after changing the basis to the columns of V

Question 7

The characteristic polynomial

Answer 7

The characteristic polynomial of a d×d matrix A is the degree-d polynomial in λ obtained by expanding det(A − λI).

Question 8

Cayley-Hamilton Theorem

Answer 8

Let A be any matrix with characteristic polynomial f(λ) = det(A − λI). Then, f(A) evaluates to the zero matrix

Question 9

Similar matrices

Answer 9

Two matrices A and B are said to be similar when B = VAV⁻¹

Question 10

Jordan normal form

Answer 10

“almost” diagonalizes the matrix A with an upper-triangular matrix U

Question 11

Spectral Theorem

Answer 11

Let A be a d × d symmetric matrix with real entries. Then, A is always diagonalizable with real eigenvalues and has orthonormal, real-valued eigenvectors. In other words, A can be diagonalized in the form A = VΔV^T with orthogonal matrix V

Question 12

A-Orthogonality

Answer 12

Question 13

Positive Semidefinite Matrices

Answer 13

A symmetric matrix is positive semidefinite if and only if all its eigenvalues are non-negative

Question 14

Cholesky factorization

Answer 14

The Cholesky decomposition is a special case of LU decomposition, and it can be used only for positive definite matrices

Question 15

Computing A^k

Answer 15

Question 16

Similarity Matrix

Answer 16

Question 17

Similarity matrix decomposition

Answer 17

Question 18

Gaussian kernel

Answer 18

Here, σ is a parameter that controls the sensitivity of the similarity function to distances between points. Such a similarity function is referred to as a Gaussian kernel.

Question 19

Covariance Matrix

Answer 19

Question 20

Convex and concave functions definition

Answer 20

Question 21

Semidefinite positive/negative matrix relation to convexity

Answer 21

• Functions in which A is positive semidefinite correspond to convex functions, which take the shape of a bowl with a minimum but no maximum.
• Functions in which A is negative semidefinite are concave, and they take on the shape of an inverted bowl.

Question 22

Most general form of a quadratic function in multiple variables

Answer 22

Question 23

f(x, y) relation to the vertex form

Answer 23

Question 24

Additively Separable Functions

Answer 24