Chapter 10

Question 1

What is cross-validation?

Answer 1

In an attempt to estimate the true generalization error of an model on a set of data, the data-set is split up in different sets of trai9ning and test data, and paramter estimation and training and test error is calculated.

Question 2

What is the generalization error?

Answer 2

A measure of how well our model performs on average assuming that we have infinite test sets.

Question 3

Hold-out cross validation:

Answer 3

Just 1 iteration where test and training are split up and the generalization error is assumed to be equal to the test error.

Question 4

K-fold-CV:

Answer 4

Dataset split up K times into training and test data with N/K observations in each.

Question 5

What is forward selection?

Answer 5

Start by having 0 attributes. Then add attribute that minimizes generalization error the most (cross-validation). Then add another variable and do cross validation again - stop when generalization error does not decrease anymore.

Question 6

What is backwards selection?

Answer 6

start with full model and remove stuff.