Model selection and validation

Question 1

What is Model selection?

Answer 1

Is the task to choose the best algorithm or parameters for our machine learning task

Question 2

What is validation? Describe the validation set and the validation error. How large is the difference beetween the validation error and the generalization error?

Answer 2

5 / 5

Question 3

How does validation for model selection works? What the model selection curve plots? What happens when the training error decreases but the validation error increases?

Answer 3

5 / 7-9

Question 4

how can we estimate the true risk after model selection? Train-Validation-Test Split

Answer 4

5 / 11

Question 5

How k-fold cross validation works? And when is it generally used?

Answer 5

5 / 12

Question 6

Define the leave-one-out cross validation

Answer 6

5 / 12

Question 7

How can you improve your model (what if learning fails)?

Answer 7

5 / 14-20

Question 8

Is the validation error a good estimation of the true risk when r is large?

Answer 8

5 / 10