Week 1

Question 1

Linear modelling

Answer 1

Learning a linear relationship between attributes and responses.

Question 2

What does this mean?
t = f(x;a)

Answer 2

A function f() that acts on x and has a parameter a.

Question 3

What parameter is known as the intercept?

Answer 3

w0 in
f(x) = w0 + w1*x

Question 4

What does the squared loss function describe?

Answer 4

How much accuracy we are losing through the use of a certain function to model a phenomenon.

Question 5

What is this?
Ln(y, f(x;w0,w1))

Answer 5

The squared loss function, telling us how much accuracy we lose through the use of f(x;w0,w1) to model y.

Question 6

How do you calculate the average loss across a whole dataset?

Answer 6

L =
1/N *
SUM(n=1 to N) of the squared loss function with xn.

Question 7

argmin means…

Answer 7

Find the argument that minimises.

Question 8

Bias-variance tradeoff

Answer 8

The tradeoff between a model’s ability to generalise and the risk of overfitting.

Question 9

Validation set

Answer 9

A second dataset that is used to validate the predictive performance of the model.

Question 10

K-fold cross-validation

Answer 10

Splits the data into K equally sized blocks. Each block is a validation set with the other K-1 blocks as training set.

Question 11

LOOCV (abbreviation)

Answer 11

Leave-One-Out Cross-Validation

Question 12

What is LOOVCV?

Answer 12

A type of K-fold cross-validation where K=N.

Question 13

0! ==

Answer 13

1

Question 14

What is a prerequisite for multiplying an n x m matrix A and a q x r matrix B?

A*B is possible if…

Answer 14

m == q
So the number of columns of the first matrix needs to be equal to the number of rows in the second matrix.

Question 15

(X*w)^T can be simplified to…

Answer 15

(w^T) * (X^T)

Question 16

(ABCD)^T can be simplified to…

Answer 16

( (AB) (CD) )^T

Question 17

( (AB) (CD) )^T can be simplified to…

Answer 17

(CD)^T * (AB)^T

Question 18

(CD)^T * (AB)^T can be simplified to…

Answer 18

D^T * C^T * B^T * A^T

Question 19

What is the partial derivative with respect to w of:
w^T * x

Answer 19

x

Question 20

What is the partial derivative with respect to w of:
x^T * w

Answer 20

x

Question 21

What is the partial derivative with respect to w of:
w^T * w

Answer 21

2w

Question 22

What is the partial derivative with respect to w of:
w^T * c*w

Answer 22

2cw

Question 23

Multiplying a scalar by an identity matrix results in..

Answer 23

a matrix with the scalar value on each diagonal element.

Question 24

The inverse of a matrix that only has values on the diagonal, is

Answer 24

Another diagonal matrix where each diagonal element is the inverse of the corresponding element in the original.

Question 25

How do we write the optimum value of w?

Answer 25

^w
w met een dakje erop

Question 26

^w ==

(The formula)

Answer 26

(X^T * X) ^-1 * X^T * t

Question 27

Linear regression

Answer 27

Supervised learning where t is a real number.

Question 28

Linear classification

Answer 28

Supervised learning where t is an element from a finite set.

Question 29

In supervised learning, each dataset is of the form x,t (with t in R in regression). What is the goal?

Answer 29

We look for a hypothesis such that t = f(x). We want the computer to predict the data.

Question 30

Overfitting

Answer 30

When the model comes up with a hypothesis that is too complex, so it fits the existing data very well but has terrible prediction skills.

Question 31

What does regularisation add to supervised learning?

Answer 31

It doesn’t only try to find minimized loss, but also to find minimal weights. So a penalty is added for a higher weight.

Question 32

What is the central question in the generative modelling problem?

Answer 32

Can we build a model that could generate a dataset like ours?

Question 33

What does the equation
f(x;w) = w^T * x
do?

Answer 33

It generates a datapoint (the label essentially) for every input data info x.

Question 34

What does the italics N mean?
Example: N(0, sig^2)

Answer 34

It means ‘normal distribution with mean 0 and variance sig^2’.

Question 35

For a Gaussian variable, the most likely point corresponds to…

Answer 35

the mean

Question 36

Give the name (left side) of the function for the joint density of t over all datapoints in a dataset.

Answer 36

p(t | x, w, sig^2)

Question 37

p(t | x, w, sig^2) =

Answer 37

PRODUCT(n=1 to N) p(t.n | x.n, w, sig^2)

Question 38

Give the formula for log L, the log likelihood:

Answer 38

-(N/2) * log(2pi) -
N * log(sig) -
(1/(2 * sig^2)) *
SUM(n=1 to N) of (t.n - w^T * n)^2

Question 39

Give the Bernoulli distribution

Answer 39

P(X=x) = q^x * ((1-q)^(1-x))

Question 40

IID (abbreviation)

Answer 40

independent and identically distributed

Question 41

When is a matrix A negative definite?

Answer 41

If x^T * A * x < 0 for all vectors of real values x.

Question 42

How do we actually show negative definiteness?

Answer 42

By solving the equation
- (1/sig^2) * z^T * X^T * Xz <0
for any vector z that z^T * X^T * Xz > 0

Question 43

What is M with a line on it?

Answer 43

The expected value of the squared error between estimated parameter values and the true values.

Question 44

Give the formula for M with the stripe on it.

Answer 44

M_ = B^2 + V
with B= bias and V = variance

Question 45

A function of a random variable is…

Answer 45

itself a random variable.

Question 46

How can you check if two random variables x and y are independent?

Answer 46

Check if p(x, y ) = p(x)*pp(y)

Question 47

How do you check conditional independence?

Answer 47

p(x,y|z) = p(x|z) * p(y|z)

Question 48

If you know the probability of the outcomes of a random variable X, how do you calculate the expected value E(X)?

Answer 48

Multiply each value of X by its probability and add all these products.

E(X) = SUM of all ( x * P(X=x))

Question 49

What should you mention when multiplying probabilities of variables?

Answer 49

That the variables are independent.

Question 50

How do you calculate the variance of a random variable?

Answer 50

Use the formula
var(X) = E(X^2) - E(X)^2.

Question 51

How do you calculate E(X^2)?

Answer 51

The probabilities in E(X) stay the same, but you now need to use the square of x instead of x to calculate the sum of probabilities.

Question 52

What can you say about the hypothesis/ the weight vector in a model with data in matrix X with N rows and 1 column?

Answer 52

The matrix usually contains feature values in the columns, and an additional column per row with a 1. Thus there are no features in this matrix X.
The weight vector contains a value for every column, now only 1s so w.T * x.n = w.0 * 1 = w.0, it’s just a number that is multiplied with every xn and gives the same hypothesis h for every value xn.

Question 53

What is the loss function of the least-squares regression problem?

Answer 53

1/N * SUM(n=1 to N) (t.n-w.T*x.n)^2)

Question 54

The derivative of a sum of terms is equal to…

Answer 54

the sum of the derivatives of those terms.

Question 55

The derivative of (ax+b)^q = …

Answer 55

(q(ax+b)^q-1) * a

Question 56

What would the log of the likelihood be:

PRODUCT(n=1 to N) of r^x.n * ((1-r)^(1-x.n))

Answer 56

SUM(n=1 to N) of x.nlog(r) + (1-x.n)log(1-r)

Question 57

When you are asked to compute the maximum likelihood estimate of a Bernoulli parameter, what do you do?

Answer 57

Take the derivative of the log likelihood, so a log L/a r. If there is a sum in the log L, it stays in the derivative for now. All a*log(b) become a/b in the derivative.
Equate it to zero.

Question 58

Multiplying a negative definite matrix by a negative constant gives…

Answer 58

a positive definite matrix.