Week 2

Question 1

What do we call the Gaussian distribution if we generalise it to define a density function over continuous vectors?

Answer 1

Multivariate Gaussian distribution.

Question 2

How do we define the multivariate Gaussian density for a vector x = [x.1, …, x.D]^T ?

Answer 2

p(x) = (1 / ( (2*pi)^(D/2) |SIG| ^0.5)) * exp { -0.5 * (x-mu)^T * SIG^-1 * (x-mu) }

Question 3

What is the mu in the formula for the multivariate Gaussian density?

Answer 3

The mean, a vector of the same size as vector x.

Question 4

What does the d-th element of mu tell us in the formula for multivariate Gaussian density?

Answer 4

The mean value of x.d

Question 5

What is the form of the variance, the SIG, in the formula for multivariate Gaussian density?

Answer 5

a DxD covariance matrix

Question 6

I^-1 ==
(identity matrix)

Answer 6

I

Question 7

I * matrix ==
(identity matrix I)

Answer 7

matrix

Question 8

The exp of a sum gives the same result as…

Answer 8

the product of exps

Question 9

|A| is the … of matrix A

Answer 9

determinant

Question 10

How do you calculate the determinant of a 2x2 matrix A:
[ a b
c d ]

Answer 10

|A| = ad - bc

Question 11

What is |I|?

Answer 11

1

Question 12

(2*pi)^(D/2) can be written as…

Answer 12

PRODUCT(d=1 to D) of (2*pi)^(1/2)

Question 13

What is Tr(A) for matrix A?

Answer 13

the trace of a square matrix A, the sum of the diagonal elements of A

Question 14

If A = I.D, so the DxD identity matrix, then Tr(I.D) =

Answer 14

SUM(d=1 to D) 1 = D

Question 15

Tr(AB) ==

Answer 15

Tr(BA)

Question 16

Tr(w^T * w) ==

Answer 16

w^T * w,
since w^T * w results in a scalar.

Question 17

What does the binomial distribution describe?

Answer 17

The probability of a certain number of successes in N binary events.

Question 18

How do you calculate the probability of y successes in N tosses with probability r per toss and a binary event?

Answer 18

P(Y = y ) = (N over y) * r^y * (1-r)^(N-y)

Question 19

When is a likelihood-prior pair conjugate?

Answer 19

If it results in a posterior of the same form as the prior.

Question 20

Suppose we have data generated by a model with distribution t ~N(X.w, sig^2). What does this say about X.w?

Answer 20

X.w is the mean vector, it gives the t we should get for every x-element. Every element in X.w is a Gaussian random variable, but X.w as a whole is a multivariate Gaussian.

Question 21

What can you do in the equation
E[w^] = E[ (X^T * X) ^-1 * X^T * t] if it’s a linear function?

Answer 21

You can change the order in which you take the expectation and apply the linear function, so:
= (X^T * X) ^-1 * X^T * E[t].

Question 22

What is the expectation of a multivariate normal distribution?

Answer 22

The mean.

Question 23

Why do (X^T * X)^-1 and (X^T * X) cancel each other out?

Answer 23

They are inverses of each oter.

Question 24

When do we call an estimator x unbiased?

Answer 24

When it has the property that
E[x^] = x.

Question 25

cov[w^] ==

Answer 25

sig^2 * (X^T * X) ^-1

Question 26

The cov[w^] matrix is the inverse of…

Answer 26

the Hessian matrix of 2nd partial derivatives.

Question 27

What does a large covariance mean in the cov[w^] matrix?

Answer 27

That we’re uncertain.

Question 28

In linear regression, if there is a negative covariance between w.0^and w.1^, then

Answer 28

there is a dependence such that if one goes up, the other goes down.

Question 29

P(Y.N = y.N | R=r) ==

Answer 29

(N over y.N) * r^y.N * ((1-r)^(N-y.N))

Question 30

What do we need to compute the joint distribution of r and y p(r,y.N)?

Answer 30

P(Y.N = y.N | R=r) and p(r). So the conditional distribution of Y.N given R and the density of r, the prior.

Question 31

p(r|Y.N = y.N ) ==

Answer 31

p(r,y.N) / P(Y.N = y.N)

Question 32

p(r, y.N)

Answer 32

P(Y.n=y.N | R=r) * p(r)