Introduction

Question 1

How is defined a learning task?

Answer 1

By a performance P, a task T and an experience E : <P, T, E>.

Question 2

What is a linear model?

Answer 2

It’s a model which is linear in terms of the parameters.

Question 3

How to solve overfitting?

Answer 3

-train on more data
-regularization

Question 4

What is regularization?

Answer 4

It consists in adding a penalty term to the error function to discourage coefficients from reaching large values:
E(w) = […] + λ/2 • ||w||².

Question 5

What is model selection?

Answer 5

It’s a procedure to choose the value of an hyper-parameter λ:
1. For different values of λ:
- train the model
- compute the performance on the validation set
2. Choose the value of λ that has the best validation performance.
3. Compute the test performance for the model with chosen value of λ.

Question 6

What is cross-validation?

Answer 6

It’s splitting the training set into a training set and a validation set for model selection.

Question 7

What is k-fold cross-validation?

Answer 7

k-fold cross-validation consists in dividing the train set in k folds and doing the training and validation steps using iteratively each of the folds for validation (while using the k-1 others for training). All performance are then average. This ensures a statistically representative training/validation set split but is very costly.

Question 8

What is the Machine Learning Pipeline?

Answer 8

1. Define the input and output.
2. Collect examples for the task.
3. Divide the examples into training, validation and testing sets.
4. Do data preprocessing.
5. Define your model (including parameters and hyper-parameters).
6. Define the error and loss functions you want to minimize.
7. For different values of hyper-parameters:
   
   • learn model parameters by minimizing the loss function
   • compute validation performance
8. Pick the best model based on validation performance.
9. Test the model on the testing set.

Question 9

What is the assumption of a Least squares classifier?

Answer 9

The decision boundary is linear.

Question 10

What is the assumption of a k-Nearest-Neighbors (k-NN) classifier?

Answer 10

The class distribution is locally smooth.

Question 11

Compare linear regression and k-NN regression in terms of:
-dependance to the model
-learning tasks
-prediction speed
-smoothness of the solution
-stability of the solution
-inductive bias
-bias, variance

Answer 11

linear regression VS k-NN regression:
model-based VS model-free
learning of parameters VS no learning
fast VS slow
smooth VS not smooth
more stable VS less stable
f(x) is well approximated by a globally linear function VS f(x) is well approximated by a locally constant function
high bias, low variance VS low bias, high variance