Session 4.2

Question 1

Higher number of folds means…

Answer 1

having to run more models, having larger train sets and smaller test sets

10 folds are the most common option. 5 folds are also frequently used.

Question 2

Having larger training sets leads to…

Answer 2

better performance in each model

Question 3

Having smaller test sets leads to

Answer 3

higher variance across models

Question 4

The Cumulative Response curve

Answer 4

plots the true positive rate as a function of the percentage of test instances targeted

Question 5

Underfitting

Answer 5

A model that is too simple does not fit the data well (high bias)

e.g., fitting a quadractic function with a linear model

Question 6

Overfitting

Answer 6

A model that is too complex fits the data too well (high variance)

e.g., fitting a quadractic function with a 3rd degree function

Question 7

Bias

Answer 7

a model that underfits is wrong on average (high bias) but is not highly affected by slightly different training data

Question 8

Variance

Answer 8

a model that overfits is right on average, but is highly sensitive to specific training data

Question 9

Ensemble methods

Answer 9

use multiple algorithms to obtain better predictive performance than could be obtained from any of the algorithms by itself

Question 10

Using multiple algorithms usually increases model performance by:

Answer 10

reducing variance: models are less dependent on the specific training data

Examples:

• Bagging (or bootstrap aggregation)
• Random Forest
• Boosting