Linear Models

Question 1

How does Ridge Regression work?

Answer 1

Ridge regression (aka regularized least squares) adds a regularization parameter to penalize overly complex models.
The regularization parameter lambda can be chosen with cross-validation

Question 2

How complicated functions can ridge regression learn?

Question 3

What does the objective function of ridge regression measure?

Answer 3

(objective function is the target that your model tries to optimize) Shrinkage method. The objective is to shrink some of the parameters to zero.

Question 4

How do you control model complexity in Ridge Regression?

Answer 4

ith the hyper parameter lambda.

Question 5

How does a linear model work, what does it mean if a coefficient in the learned w-vector is large/small/zero?

Answer 5

zero means no relationship, + value positive relationship, etc. 1.0 is a perfect correlation.

Question 6

What kind of algorithm do you need for training ridge regression?

Question 7

Name other linear regression/classification methods.

Answer 7

lasso regression (good for feature selection)
Elastic net
Support Vector regression

Question 8

What’s a linear model?

Answer 8

The linear model, a simple yet popular choice:
f (x) = w1 · x1 + … + wd · xd + b
▶ x1, …, xd , feature values
▶ w1, …,wd model coefficients
▶ b ∈ R intercept term

Question 9

How to find the regression line in a single feature case?

Answer 9

you try to find a line or non-linear function where the mean square error in the y direction is minimized (sum of squared errors in the y direction)

Question 10

What is the downside of the classical least squares method (Gaussian) and what is the solution?

Answer 10

Works fine in smaller dimensions, prone to overfitting in high dimensions. Sensitive to outliers. Solution: ridge regression.