Lecture 11

Question 1

What is the observation vector?

Answer 1

the vector of observed values

Question 2

What is the hypothesis vector?

Answer 2

the vector of predicted values

Question 3

What is the error vector?

Answer 3

the vector of signed errors

Question 4

What is the equation for the error vector?

Answer 4

e_i = y_i - H(x_i)

Question 5

How do we rewrite the mean squared error of H?

Answer 5

1/n ||y - h||^2

Question 6

How does the linear hypothesis function H(x) = w_0 + w_1x be written as?

Answer 6

a matrix of n x 1 where each line is w_0 + w_1x_1, w_0 + w_1x_2, etc.

Question 7

What is the notation for the design matrix?

Answer 7

X E R^nx2

Question 8

What does the design matrix look like?

Answer 8

a matrix of n x 2 where the first column is all 1’s and the second column is n

Question 9

What is the parameter vector’s notation?

Answer 9

w E R^2

Question 10

What are the variables in the parameter vector?

Answer 10

w_0 and w_1

Question 11

What is the mean squared error for the design matrix?

Answer 11

R_sq(w) = 1/n ||y - Xw ||^2

Question 12

What is the equation for the optimal parameter vector?

Answer 12

w* = [w_0* w_1*]^T

Question 13

What is the minimizer of R_sq(w)?

Answer 13

argmin||e||

Question 14

What do we need to know about X^TX in order for it to be invertible?

Answer 14

it has to be full rank

Question 15

What does full rank mean?

Answer 15

all columns are linearly independent

Question 16

What happens if X^TX is not full rank?

Answer 16

there are infinitely many solutions to the normal equations

Question 17

What is the input w* to error(w)?

Answer 17

the one that satisfies the normal equations

Question 18

What is the unique solution if X^TX is invertible?

Answer 18

w* = (X^TX)^-1X^Ty

Question 19

What is a function of a vector?

Answer 19

it is a function of multiple variables, which are the components of the vector

Question 20

What is the gradient of R_sq(w) with respect to w?

Answer 20

the vector of partial derivatives where w_0, w_1, …, w_d are the entries of the vector w

Question 21

What is the equivalent equation for R_sq(w) = 1/n ||y - Xw||^2?

Answer 21

1/n (y - Xw)^T (y - Xw)

Question 22

What does (AB)^T equal?

Answer 22

B^T A^T

Question 23

What does d/dw (y * y) =?

Answer 23

0