Week 1

Question 1

Wat zijn de 5 onderdelen van een leerprobleem?

Answer 1

1) Input x in X.
2) Output y in Y.
3) Doelfunctie f: X -> Y.
4) De dataset met (x1, y1)…
5) De geleerde hypothese g: X -> Y.

Question 2

Wanneer is ML toepasbaar op een probleem? Geef 3 voorwaarden.

Answer 2

1) Er is een patroon.
2) Het lukt niet om dit patroon te beschrijven door het probleem te analyseren.
3) Er zijn gegevens waar we uit kunnen leren.

Question 3

What is H in ML?

Answer 3

The hypothesis set of candidate formulas that are under consideration.

Question 4

What is the role of h(x)?

Answer 4

A functional form that assigns the weights to the different components of the input vector.

Question 5

When is a dataset linearly separable?

Answer 5

There is a choice for the parameters that classifies all the training examples correctly.

Question 6

PLA

Answer 6

perception learning algorithm

Question 7

What is the goal of the perceptron learning algorithm?

Answer 7

Finding a hypothesis that classifies all the data points in data set D correctly.

Question 8

Supervised learning setting

Answer 8

When the training data contains explicit examples of what the correct output should be for given inputs.

Question 9

Waar bestaat de hypotheseruimte uit bij k-nearest neighbors?

Answer 9

Bijna alle functies van inputs naar outputs.

Question 10

Active learning

Answer 10

The data set is acquired by the learner through asking for a label for specific entries.

Question 11

What is the standard formula for h(x)?

Answer 11

h(x) = sign(w.T * x)

Question 12

Online learning

Answer 12

The data set is given to the algorithm one example at a time. Learning takes place as data becomes available.

Question 13

Transfer learning

Answer 13

When training an algorithm on data results in a model, and that model is used on a new problem or task. It uses the info learned on the first problem to improve on the second one.

Question 14

What is the update formula for w?

Answer 14

w(t+1) = w(t) + y(t)*x(t)

Question 15

Reinforcement learning

Answer 15

The training example does not contain the target output, but contains some possible output together with a measure of how good that output is.

Question 16

in-sample error

Answer 16

E.in(h): The error rate within a sample: the fraction of the data set where h and f disagree.
Example: the mistakes on a practice test.

Question 17

Wat is een pluspunt van het PLA?

Answer 17

Het doorzoekt een oneindig grote verzameling hypothesen.

Question 18

What kind of data do you need to use the Perceptron Learning Algorithm?

Answer 18

Linearly separable data.

Question 19

Give the formula for E.in(h):

Answer 19

In-sample error:

= 1/N * the amount of datapoints in the sample where h(x) and f(x) disagree.

Question 20

What does the out-of-sample error denote?

Answer 20

How accurately the hypothesis function performs on data it hasn’t seen before.
Example: performance on exam.

Question 21

What is the deterministic answer to ‘Does the data set D tell us anything outside of D that we didn’t know before?’?

Answer 21

No.
D tells us something certain about f outside of D.

Question 22

What is the probabilistic answer to ‘Does the data set D tell us anything outside of D that we didn’t know before?’?

Answer 22

Yes.
D tells us something likely about f outside of D.

Question 23

What are the two questions that present the feasability of learning?

Answer 23

1) Can we make sure that Eout(g) is close enough to Ein(g)?
2) Can we make Ein(g) small enough?

Question 24

What effect does a more complex H have?

Answer 24

It gives more flexibility in finding some g that fits the data well, leading to small Ein(g).

Question 25

What effect does a complex f have?

Answer 25

We get a worse value for Ein(g).

Question 26

Noisy function

Answer 26

A function where the output is not uniquely determined by the input.

Question 27

How does neigbors-based classification learn?

Answer 27

Does not attempt to construct a general or internal model, but simply stores instances of the training data.

Question 28

Wat is mu in het model met knikkers en vazen?

Answer 28

De proportie rode knikkers in de vaas.

Question 29

What is the formula for the line that classifies the datapoints?

Answer 29

w1x1 + w2x2 + b = 0

Question 30

Wat is de onbekende in het knikkers/vazenmodel?

Answer 30

Mu, de proportie rode knikkers in de vaas.

Question 31

Unsupervised learning

Answer 31

The training data does not contain any output information at all.

Question 32

Data mining

Answer 32

A practical field that focuses on finding patterns.

Question 33

What is nu in the marbles-vases model?

Answer 33

The fraction of red marbles within a random sample of N marbles that you pick from the vase.

Question 34

What does the Hoeffding Inequality denote?

Answer 34

The maximum probability that the sum of the bounded independent random variables is different from its expected value more than a certain amount x.
Example: for a fixed N, a higher precision of the difference between E.in & E.out makes the probability of that difference lower.

Question 35

Why is the Hoeffding Inequality important for machine learning?

Answer 35

Through the Hoeffding Inequality, learning, generalizing to unknown data, is made possible without knowing the target function.
This is because neither nu nor mu are needed to bound the probability in the H.I.

Question 36

What is the Hoeffding Inequality used for?

Answer 36

It quantifies the relationship between nu and mu.

Question 37

Give the Hoeffding Inequality:

Answer 37

P[|nu-mu| > epsilon] <_ 2e ^ (-2*epsilon ^2 *N)

The probability that the difference between the fraction of red marbles in the random sample and the actual fraction of red marbles in the vase is bigger than epsilon,
is smaller than or equal to the right-side.
Epsilon is a positive value we choose, how much nu and mu can be different.

Question 38

What is always true about E.in(h), E.out(h) and epsilon?

Answer 38

|E.in(h) - E.out(h)| > epsilon

Question 39

If event B1 implies event B2, thus
B1 -> B2,
then…

Answer 39

The probability of event B1 is smaller or equal to the probability of event B2, thus

P[B1] <_ P[B2]

Question 40

Describe the nearest-neigbors algorithm:

Answer 40

Given an input, search for the closest input we have seen and copy that output.