Regression

Question 1

What does regression minimize?

Answer 1

Regression minimizes the sum of squared errors

Question 2

How could you deal with outliers without changing their values?

Answer 2

First you train your model.
Second you remove around 10% of points with the largest error.
And then you retrain.

This would converge to a better model.

Question 3

Is feature scaling relevant for linear regression

Answer 3

It is just if it’s a regularized regression. In a normal regression the betas will change acording to the transformations made to the variable values.

In a regularized regression, standardization is important, because we loose this invariance we have with least squares. This is easy to see: both lasso and ridge do regularization based on the size of the betas, so any transformation which change the relative sizes of the betas will change the result!