Bias, Variance, Underfitting, Overfitting

Question 1

Overfitting

Answer 1

Occurs when the model fits the data too well and captures the noise of the data. It then becomes difficult for the model to generalize to new examples that were not in the training set.

The model shows low bias and high variance.

Often the result of a too complex model.

Using more data is the simplest and best possible way to prevent overfitting

Question 2

Underfitting

Answer 2

Occurs when a model cannot capture the underlying trend of the data and does not fit the data well enough.

The model shows low variance but high bias.

Often a result of an excessively simple model.

Question 3

Bias

Answer 3

“Bias is the algorithm’s tendency to consistently learn the wrong thing by not taking into account all the information in the data (underfitting).”

Question 4

Variance

Answer 4

“Variance is an error from the sensitivity to small fluctuations in the training set. High variance can cause an algorithm to model the random noise in the training data, rather than the intended outputs (overfitting).”