Lecture 10

Question 1

What is an n x d matrix?

Answer 1

a table of numbers with n rows and d columns

Question 2

How do we denote matrices?

Answer 2

using upper-case letters

Question 3

How do we say write a matrix A that has two rows and three columns?

Answer 3

AER^2*3

Question 4

What is another way to think of a matrix?

Answer 4

as several column vectors, stacked next to each other

Question 5

When can we add two matrices?

Answer 5

only if they have the same dimensions

Question 6

How does addition occur between two matrices?

Answer 6

elementwise

Question 7

How does scalar multiplication occur?

Answer 7

elementwise

Question 8

When can we multiply matrices A and B?

Answer 8

if and only if # of columns in A = # of rows in B

Question 9

If A is n * d and B is d * p then what is AB?

Answer 9

n * p

Question 10

What is the notation that shows multiplication is distributive in matrices?

Answer 10

A(B + C) = AB + AC

Question 11

What is the notation that shows multiplication is associative in matrices?

Answer 11

(AB)C = A(BC)

Question 12

What is the notation that shows the transpose of sum in matrices?

Answer 12

(A + B)^T = A^T + B^T

Question 13

What is the notation that shows the transpose of product in matrices?

Answer 13

(AB)^T = B^TA^T

Question 14

What is one way of thinking about the product Av?

Answer 14

it is the dot product of v with every row of A

Question 15

What is another way of thinking about the product Av?

Answer 15

it is a linear combination of the columns of A, using the weights in v

Question 16

What is the span of the columns of X?

Answer 16

consists of all vectors that can be written in the form Xw

Question 17

What is the condition that e must be orthogonal to each column of X equivalent to?

Answer 17

the condition that X^Te = 0

Question 18

What is the normal equations?

Answer 18

X^TXw = X^Ty

Question 19

What can we assume if X^TX is invertible?

Answer 19

the vector w* = (X^TX)^-1X^Ty and requires X^TX to be full rank

Question 20

What happens if X^TX is not full rank?

Answer 20

then there are infinitely many solutions to the normal equations

Question 21

What is the observation vector?

Answer 21

yER^n; vector of observed “actual values”

Question 22

What is the hypothesis vector?

Answer 22

hER^n with components H(x_i); the vector of predicted values

Question 23

What is the error vector?

Answer 23

the vector eER^n with components: e_i = y_i - H(x_i)

Question 24

What is the mean squared error of H rewritten?

Answer 24

1/n ||y - h||^2

Question 25

How do we define the design matrix?

Answer 25

XER^n*2

Question 26

How do we define the parameter vector?

Answer 26

wER^2