Introduction Flashcards
Column vector
A d × 1 matrix
Row vector
A 1 × d matrix
The dot product is a commutative operation
Squared norm or Euclidean norm
Lp-norm
The (squared) Euclidean distance between x and y
Dot products satisfy the Cauchy-Schwarz inequality, according to which the dot product between a pair of vectors is bounded above by the product of their lengths
The cosine function between two vectors
The cosine law
You are given the orthonormal directions [3/5, 4/5] and [−4/5, 3/5]. One can represent the point [10, 15] in a new coordinate system defined by the directions [3/5, 4/5] and [−4/5, 3/5] by computing the dot product of [10, 15] with each of these vectors.
Therefore, the new coordinates [x’, y’] are defined as follows:
x’ = 10 ∗ (3/5) + 15 ∗ (4/5) = 18
y’ = 10 ∗ (−4/5) + 15 ∗ (3/5) = 1
One can express the original vector using the new axes and coordinates as follows:
[10, 15] = x’[3/5, 4/5] + y’[−4/5, 3/5]
An example of a multiplication of a 3×2 matrix A = [aij] with a 2-dimensional column vector
x = [x1, x2]T
An example of the multiplication of a 3-dimensional row vector v = [v1, v2, v3] with the 3 × 2 matrix A
The multiplication of an n×d matrix A with a d-dimensional column vector x to create an n-dimensional column vector Ax is a weighted sum
Outer Product
The outer product is not commutative; the order of the operands matters
An example of a matrix multiplication
Types of matrices
Inverse of a 2 x 2 matrice
An orthogonal matrix is a square matrix whose inverse is its transpose
Infinite geometric series of matrices
Frobenius Norm (energy)
Defined as the square root of the sum of the absolute squares of the elements of a matrix
The trace of a square matrix A
Is defined by the sum of its diagonal entries
The energy of a rectangular matrix A is equal to the trace of either AAT or AT A
Pre-multiplying a matrix X with an elementary matrix corresponding to an interchange results in an interchange of the rows of X
Post-multiplication of matrix X with the following elementary matrices will result in exactly analogous operations on the columns of X to create X’
Permutation matrix and its transpose
Are inverses of one another because they have orthonormal columns
The point [a cos(α), a sin(α)] has magnitude a and makes a counter-clockwise angle of α with the X-axis. One can multiply it with the rotation matrix shown here to yield a counter-clockwise rotation of the vector with angle θ
Elementary matrices for geometric operations
A fundamental result of linear algebra is that any square matrix can be shown to be a product of rotation/reflection/scaling matrices
By using a technique called singular value decomposition
Matrix factorization using the squared Frobenius norm
The squared Frobenius norm is the sum of the squares of the entries in the residual matrix
(D−UVT)
The d-dimensional vector of partial derivatives is referred to as the gradient
Univariate Taylor Expansion of the function w at a
Taylor expansion of the exponential function at 0
exp(w)
Multivariable Taylor expansion of functions F(w) with d-dimensional arguments of the form w = [w1 …wd]T. The Taylor expansion of the function F(w) about w = a = [a1 …ad]T can be written as follows:
Second-order Taylor approximation can be written in vector form
Normal equation using calculus
Gradient descent with the normal equation
Gradient descent in compact form
Transpose of AB
The product of matrices AB can be expressed as the outer product of the columns of A and the rows of B
