E8

Question 1

Underfitting

Answer 1

A model that is too simple does not fit the data well (high bias)

e.g., fitting a quadractic function with a linear model

Question 2

Overfitting

Answer 2

A model that is too complex fits the data too well (high variance)

e.g., fitting a quadractic function with a 3rd degree function

Question 3

Bias

Answer 3

a model that underfits is wrong many times (high bias) but is not highly affected by slightly different training data

Question 4

Variance

Answer 4

a model that overfits is right on average, but is highly sensitive to specific training data

Question 5

Variance-bias tradeoff

Answer 5

When trying the optimal model we are in fact trying to find the optimal tradeoff between bias and variance

Question 6

We can reduce variance by

Answer 6

putting many models together and aggregating their outcomes