Introduction

Question 1

Column vector

Answer 1

A d × 1 matrix

Question 2

Row vector

Answer 2

A 1 × d matrix

Question 3

The dot product is a commutative operation

Answer 3

Question 4

Squared norm or Euclidean norm

Answer 4

Question 5

L_p-norm

Answer 5

Question 6

The (squared) Euclidean distance between x and y

Answer 6

Question 7

Dot products satisfy the Cauchy-Schwarz inequality, according to which the dot product between a pair of vectors is bounded above by the product of their lengths

Answer 7

Question 8

The cosine function between two vectors

Answer 8

Question 9

The cosine law

Answer 9

Question 10

You are given the orthonormal directions [3/5, 4/5] and [−4/5, 3/5]. One can represent the point [10, 15] in a new coordinate system defined by the directions [3/5, 4/5] and [−4/5, 3/5] by computing the dot product of [10, 15] with each of these vectors.

Answer 10

Therefore, the new coordinates [x’, y’] are defined as follows:

x’ = 10 ∗ (3/5) + 15 ∗ (4/5) = 18

y’ = 10 ∗ (−4/5) + 15 ∗ (3/5) = 1

One can express the original vector using the new axes and coordinates as follows:

[10, 15] = x’[3/5, 4/5] + y’[−4/5, 3/5]

Question 11

An example of a multiplication of a 3×2 matrix A = [a_ij] with a 2-dimensional column vector

x = [x₁, x₂]^T

Answer 11

Question 12

An example of the multiplication of a 3-dimensional row vector v = [v₁, v₂, v₃] with the 3 × 2 matrix A

Answer 12

Question 13

The multiplication of an n×d matrix A with a d-dimensional column vector x to create an n-dimensional column vector Ax is a weighted sum

Answer 13

Question 14

Outer Product

Answer 14

Question 15

The outer product is not commutative; the order of the operands matters

Answer 15

Question 16

An example of a matrix multiplication

Answer 16

Question 17

Types of matrices

Answer 17

Question 18

Inverse of a 2 x 2 matrice

Answer 18

Question 19

An orthogonal matrix is a square matrix whose inverse is its transpose

Answer 19

Question 20

Infinite geometric series of matrices

Answer 20

Question 21

Frobenius Norm (energy)

Answer 21

Defined as the square root of the sum of the absolute squares of the elements of a matrix

Question 22

The trace of a square matrix A

Answer 22

Is defined by the sum of its diagonal entries

Question 23

The energy of a rectangular matrix A is equal to the trace of either AA^T or A^T A

Answer 23

Question 24

Pre-multiplying a matrix X with an elementary matrix corresponding to an interchange results in an interchange of the rows of X

Answer 24

Question 25

Post-multiplication of matrix X with the following elementary matrices will result in exactly analogous operations on the columns of X to create X'

Answer 25

Question 26

Permutation matrix and its transpose

Answer 26

Are inverses of one another because they have orthonormal columns

Question 27

The point [a cos(α), a sin(α)] has magnitude a and makes a counter-clockwise angle of α with the X-axis. One can multiply it with the rotation matrix shown here to yield a counter-clockwise rotation of the vector with angle θ

Answer 27

Question 28

Elementary matrices for geometric operations

Answer 28

Question 29

A fundamental result of linear algebra is that any square matrix can be shown to be a product of rotation/reflection/scaling matrices

Answer 29

By using a technique called singular value decomposition

Question 30

Matrix factorization using the squared Frobenius norm

Answer 30

The squared Frobenius norm is the sum of the squares of the entries in the residual matrix (D−UV^T)

Question 31

The d-dimensional vector of partial derivatives is referred to as the gradient

Answer 31

Question 32

Univariate Taylor Expansion of the function w at a

Answer 32

Question 33

Taylor expansion of the exponential function at 0 exp(w)

Answer 33

Question 34

Multivariable Taylor expansion of functions F(w) with d-dimensional arguments of the form w = [w¹ ...w^d]^T. The Taylor expansion of the function F(w) about w = a = [a¹ ...a^d]^T can be written as follows:

Answer 34

Question 35

Second-order Taylor approximation can be written in vector form

Answer 35

Question 36

Normal equation using calculus

Answer 36

Question 37

Gradient descent with the normal equation

Answer 37

Question 38

Gradient descent in compact form

Answer 38

Question 39

Transpose of AB

Answer 39

Question 40

The product of matrices AB can be expressed as the outer product of the columns of A and the rows of B

Answer 40

Question 41

Answer 41