Week 3

Question 1

Wat is X in het knikker-vazen model?

Answer 1

Het aantal rode knikkers in de steekproef (het sample).

Question 2

Wat is nu, uitgedrukt in X en N?

Answer 2

X/N
Het aantal rode knikkers in de steekproef / de grootte van de steekproef.

Question 3

Geef de Hoeffding Inequality:

Answer 3

P[|E.in-E.out| > epsilon] = 2e ^(-2(epsilon^2)N)
voor alle epsilon > 0

Question 4

Noem de twee soorten supervised learning-problemen:

Answer 4

Classificatie: Y bestaat uit een klein aantal elementen (Bij binair: 2 elementen)
Regressie: Y = R

Question 5

Linear regression

Answer 5

A linear model based on the signal function. The output is the signal.

Question 6

Logistic regression

Answer 6

A linear model that outputs a probability between 0 and 1. Holds no threshold at all:
h(x) = theta * ( w^T * x )

Question 7

Give the logistic function theta(s):

Answer 7

theta(s) = (e^s) / (1+e^s)
output between 0 and 1

Question 8

Linear classification

Answer 8

Uses a hard threshold on the signal.
h(x) = sign(w^T *x)

Question 9

classification output…

Answer 9

is bounded

Question 10

regression output…

Answer 10

is real

Question 11

What is meant with the ‘soft threshold’?

Answer 11

the logistic function theta(s)

Question 12

Why is the logistic function also called a sigmoid?

Answer 12

because its shape looks like a flattened out ‘s’

Question 13

What is the target that a logistic function is trying to learn?

Answer 13

A probability that depends on the input x.

Question 14

What is the target function in logistic regression?

Answer 14

f(x) = P[y=+1 | x]

Question 15

Error measure

Answer 15

How likely it is that we would get this output y from the input x if the target distribution P(y|x) was indeed captured by our hypothesis h(x).

Question 16

Give the formula for the in-sample error in linear regression:

Answer 16

E.in(h) = 1/N * (w.T*x.i - y.i)^2

voor alle n in N

Question 17

method of maximum likelihood

Answer 17

Selects the hypothesis h(x) which maximises the probability to get all yn’s in the dataset from the corresponding xn’s

Question 18

Give the formula for the in-sample error measure for logistic regression:

Answer 18

Ein(w) = 1/N * ln(1+ e^(-yn* w.T *xn))

for all n in N.

Question 19

What is the target in linear regression?

Answer 19

A noisy target function formalized as a distribution of the random variable y.

Question 20

Linear regression: method and goal

Answer 20

We have an unknown distribution P(x,y) that generates each (xn,yn), and we want to find a hypothesis g that minimizes the error between g(x) and y with respect to that distribution.

Question 21

Matrix representation of Ein(h) in linear regression:

Answer 21

The N x (d+1) matrix with input vectors xn as rows and y the target vector as columns with yn as components/target values.

Question 22

How do you get the gradient of Ein(w) to be 0?

Answer 22

Solve the following for a w:

X^T * X * w = X^T * y

Question 23

Wat is ^y in de kwadratische fout?

Answer 23

De schatting volgens de hypothese.

Question 24

Wat is y in de kwadratische fout?

Answer 24

De correcte waarde, waar we op mikken (target waarde)

Question 25

Geef de formule voor de kwadratische fout (squared error):

Answer 25

e(^y, y) = (^y - y)^2

Question 26

Linear regression algorithm in 3 steps

Answer 26

1) Construct matrix X and vector y from the data set, with each x0=1.
2) Compute the pseudo-inverse of the matrix X.
3) Return wlin = pseudo-inverse of X * y

Question 27

OLS

Answer 27

Ordinary least squares

Question 28

hoe wordt de afgeleide van functie f in richting w.i geschreven?

Answer 28

omgekeerde a/ omgekeerde a w.i f(x)

Question 29

Wat is (omgekeerd driehoekje v) f(x)?

Answer 29

De vector van de afgeleides van f(x) in richting w0, … wn.

Question 30

Wat is de kleinste-kwadraten schatter (least-squares estimator)?

Answer 30

De oplossing wlin die je krijgt als je de gradient van E.in oplost voor gradient = 0.

Question 31

Wat is wlin?

Answer 31

w = ((X.T * X) ^-1) * X.T *y

Question 32

Wat kun je doen met de kleinste-kwadraten schatter?

Answer 32

y voorspellen voor een willekeurige x.

Question 33

Hoe voorspel je y voor een willekeurige x met de kleinste-kwadraten schatter?

Answer 33

^y = w.lin.T * x

Question 34

Wat doet de hat matrix?

Answer 34

Vertaalt de daadwerkelijke outputdata y naar outputdata die met de hypothese kloppen.

Question 35

What is the main difference between the learning approach and the design approach?

Answer 35

The role that data plays: in the design approach, the problem is well-defined and f can be analytically derived without seeing data. In the learning approach, data is needed to pin down f.