ML-03 - Regularization in regression Flashcards
ML-03 - Regularization in regression
Does underfitting mean that the model has high bias or high variance?
High bias
ML-03 - Regularization in regression
Does overfitting mean that the model has high bias or high variance?
High variance
ML-03 - Regularization in regression
How can your model respond if you have random noise in the training dataset?
Overfitting
ML-03 - Regularization in regression
If the training sample size is too small, does that cause underfitting or overfitting?
Overfitting?
ML-03 - Regularization in regression
If the number of features is too large, does that cause underfitting or overfitting?
Overfitting
ML-03 - Regularization in regression
If you simplify the assumptions in your model so that the target function is easier to approximate, can that cause overfitting or underfitting?
Underfitting.
ML-03 - Regularization in regression
How do you address overfitting? (2)
- Reduce the number of features
- Regularization (Keep features, but reduce weight values)
ML-03 - Regularization in regression
What is the formula for L2 regularization?
See image
ML-03 - Regularization in regression
What is the effect of setting lamba too high in regularization?
Weights are penalized too much, turns into a straight line (h_w(x) = w_0 + 0*w_1 + …) -> underfitting.
ML-03 - Regularization in regression
What is the effect of setting lamba too low in regularization?
No penalization effect -> overfitting
ML-03 - Regularization in regression
Describe regularized linear regression with gradient descent.
(See image)
ML-03 - Regularization in regression
Describe how to add regularization to the normal equation.
(See image)
ML-03 - Regularization in regression
Describe how to add regularization to logistic regression
(See image)
