L4_OLS

Question 1

The most popular and best understood type of regression is

Answer 1

linear regression (or ordinary least squares (OLS)) using a least-squares cost function.

Question 2

4 Statements about Linear Regression

Answer 2

… is a generic framework for prediction
… straightforwardly extends to vector labels
… can model nonlinear dependencies between data and labels
… can be made more robust (Ridge Regression)

Question 3

Gauss-Markov-Theorem

Answer 3

Under the model assumption y = w⊤ · x + ε with uncorrelated noise ε, our ordinary least squares estimator wˆ = (XX⊤)−1Xy is the Best Linear Unbiased Estimator (BLUE), i.e. the minimum variance unbiased estimator that is linear in the y.
But: in some cases biased estimators with lower variance might be more suitable

Question 4

From Classification to Regression

Answer 4

Classification, y ∈ {−1, +1}

Regression, y ∈ R