Regularization Flashcards
1
Q
What is the requirement of model parameters for regularisation?
A
They must be continuous
2
Q
What is the purpose of regularisation?
A
Penalise complexity to prevent over fitting
3
Q
What are the two methods for tuning the regularisation parameter?
A
- Validation set
- Cross validation
4
Q
How would you use a validation set to tune regularisation parameter?
A
- For 1..M train model (training set) and test (validation set)
- Choose model with best validation error
- Measure final model (test set)
5
Q
What is a good way to pick values when searching for good continuous control parameters?
A
Pick values that increase geometrically
0.01, 0.1, 0.5, 1.0, 5.0, 10.0, ….
6
Q
How does ridge regression change the linear regression error function?
A
adds lambda (regularisation parameter) times the modulus of the model parameters squared
