Linear Algebra

Question 1

What is a scalar?

Answer 1

A single number.

Question 2

What is a vector?

Answer 2

An array of numbers, arranged in an indexed order.

Question 3

How are vector variables represented?

Answer 3

Vectors have lowercase names in italic bold typeface: x

Question 4

How are scalar variables represented?

Answer 4

They are written in lowercase italics: s

Question 5

How is a set of vector indices defined?

Answer 5

S = { indices } then we write x_S

Example: S = { 1, 3, 6 } are a set of the 1,3, and 6 indexes

Question 6

How is the complement of a set indexed?

Answer 6

With a - sign:

x_-1 is the vector of all elements of x except for x₁

x_-S is the vector of all elements of x except the set S

Question 7

What is a matrix?

Answer 7

A matrix is a 2D array of numbers.

Each element is identified with two numbers.

Question 8

How are matrix variables represented?

Answer 8

With an uppercase name in bold italics: A

Question 9

How are rows and columns of a matrix indexed?

Answer 9

With a : symbol:

A_i,: represents the horizontal cross section i (row)

A_:,i represents the vertical cross section i (column)

Question 10

What is a tensor?

Answer 10

A matrix with more than two axes.

Question 11

How are tensor variables represented?

Answer 11

With a bold capital letter: A

Question 12

What is the transpose of a matrix?

Answer 12

A mirror image of the matrix across a diagonal line.

(A^T)_i,j = A_j,i

Question 13

What is the name of the line across which a transpose is mirrored?

Answer 13

main diagonal

Question 14

How is a transpose represented?

Answer 14

Superscript T:

A^T

Question 15

What is the transpose of a scalar?

Answer 15

The scalar is its own transpose.

Question 16

What is the transpose of a vector?

Answer 16

A row vector becomes a column vector and vice versa.

Question 17

How is matrix-matrix addition defined?

Answer 17

If they have the same shape, then:

C = A + B where C_i,j = A_i,j + B_i,j

Each element is added to the corresponding element.

Question 18

How is matrix-scalar addition and multiplication defined?

Answer 18

The scalar is added or multiplied to each matrix element.

This is called an element-wise operation.

Question 19

What is an element-wise matrix operation?

Answer 19

Performing the operation on each element of the matrix.

For example, in the addition of a scalar to a matrix.

Question 20

How is matrix-vector addition defined?

Answer 20

The matrix C = A + b where C_i,j = A_i,j + b_j

The vector b is added to each row of the matrix.

Question 21

What is the shape of the matrix product?

Answer 21

If A is shape m,n and B is the shape n,p then

C is the shape m,p

Question 22

What is the definition of the matrix product?

Answer 22

C_i,j = sum( A_i,k * B_k,j)

Or C_i,j as the dot product between row i of A and col j of B

Question 23

What is the distributive property of matrix multiplication?

Answer 23

A(B+C) = AB+AC

Question 24

What is the associative property of matrix multiplication?

Answer 24

A(BC) = (AB)C

Question 25

Is matrix multiplication commutative?

Answer 25

No, the condition AB = BA does not always hold.

Question 26

Is the dot product of two vectors commutative?

Answer 26

Yes

x^Ty = y^Tx

Question 27

What is the transpose of a matrix product?

Answer 27

(AB)^T = B^TA^T

Question 28

Derive the commutative rule for the dot product of two vectors.

Answer 28

x^Ty = (x^Ty)^T = y^Tx using the rule (AB)^T = B^TA^T

Question 29

What is the dot product of two perpendicular vectors and why?

Answer 29

The dot product is 0 because the projection of one vector onto the other is a point.

Question 30

What is the sign of the dot product of two vectors that point away from each other?

Answer 30

Negative

Question 31

What is the dual of a vector?

Answer 31

The linear transformation that it encodes.

Question 32

What is the dual of a linear transformation to one dimension from some space?

Answer 32

A vector in that space.

Question 33

What is the most common use of the dot product?

Answer 33

Testing if two vectors point in the same direction, or are perpendicular. It is also used for understanding projections.

Question 34

What is the conceptual nature of a vector?

Answer 34

As encoding a linear transformation in some space.

As if the vector was shorthand for a linear transformation.

Question 35

What equation describes a system of linear equations?

Answer 35

Ax = b

( A_m,1x₁ + A_m,2x₂ + … + A_m,nx_n ) = b_m

Where A is a known matrix, b is a known vector, and x is a vector of unknown variables we would like to solve for.

Each element x_i of x is one of these unknown variables.

Each row of A and each element of b provide a constraint.

Question 36

What is the identity matrix?

Answer 36

A matrix that does not change any vector when we mutiply that vector by that matrix:

I_n

Question 37

What is the matrix inverse of A?

Answer 37

The matrix inverse is the matrix A^-1 such that:

A^-1A = I_n

The matrix that when multiplied by A yields the identity.

Question 38

Solve Ax=b for x

Answer 38

1. ​Ax = b
2. A^-1Ax = A^-1b
3. I**_n**x = A^-1b because A^-1A=I_n
4. x = A^-1b because I_nx=x

Question 39

Should A^-1 be used in practice?

Answer 39

Rarely.

A^-1 can be represented only with limited precision on a digital computer. As a result, making use of the value of b can usually obtain a more accurate estimate of x.

Question 40

What is the fundamental condition required for A^-1 to exist?

Answer 40

The equation Ax=b must have exactly one solution for every value of b.

Question 41

What is a linear combination?

Answer 41

Ax = sum( x_iA_:,i )

A sum of vectors each multiplied by some corresponding scalar coefficient and adding the results.

Question 42

What is the span of a set of vectors?

Answer 42

Span of a set of vectors is the set of all points obtainable by linear combinations of the original vectors.

span( v₁, v₂, … v_n ) = { c₁v₁ + c₂v₂ + … c_nv_n | C_i∈ℝ 1<=i<=n }

Question 43

How is span used to determine if Ax=b has a solution?

Answer 43

This amounts to testing whether b is in the span of the columns of A. This is the column space or the range of A.

Question 44

In order for the system Ax=b to have a solution for all values b∈ℝ, what must be true of the column space of A?

Answer 44

The column space of A must be all of ℝ^<em>m</em>

This implies that A must have at least m columns, or n≥m

Question 45

What is linear independence?

Answer 45

A set of vectors is linearly independent if no vector in the set is a linear cominbation of the other vectors.

If we add a vector to a set that is a linear combination of the other vectors in the set, the new vector does not add any points to the set’s span.

Question 46

In order for the column space of a matrix to encompass all of ℝ^m what must be true?

Answer 46

The matrix must contain at least one set of m linearly independent columns.

(The matrix must contain a coordinate frame that defines a space ℝ^m.)

Question 47

What are the necessary and sufficient conditions for the system Ax=b to have a solution for all values b∈ℝ discoverable with the matrix inversion method?

Answer 47

A must have exactly m linear independent columns.

For A to have an inverse, A must be square (m=n) and that all of the columns be linearly independent.

Therefore, A must have m=n linearly independent columns.

Question 48

What is a singular matrix?

Answer 48

A singular matrix is square and has linearly dependent columns.