Lecture 8

Question 1

What level of variance and bias is the most fundamental (high/low)?

Answer 1

Low bias and low variance

Question 2

Which models do not perform well well by themselves due to high bias or high variance?

Answer 2

Logistic Regression
Naive Bayes
KNN
(Shallow) Decision Trees
Linear SVMs
and Kernel SVMs

Question 3

What ways can we combine models?

Answer 3

Voting
Bagging
Boosting
and Stacking

Question 4

Bagging

Answer 4

Trains many models on bootstrapped data, then takes the average

Question 5

Boosting

Answer 5

Given weak models, it runs it multiple times on reweighted training data, then lets learned classifiers vote with Gradient Boosting

Question 6

Stacking

Answer 6

Trains many models in parallel and combines them by training a meta-model to output a prediction based on the different weak models predictions.

The training data is split into two folds. One fold is used to train the base models and the other is used to train the meta-model.

Question 7

Random Forest approach

Answer 7

a bagging method where deep trees, fitted on bootstrap samples, are combines to produce an output with lower variance

Question 8

What does Bagging aim to reduce?

Answer 8

variance

Question 9

What does Boosting aim to reduce?

Answer 9

bias

Question 10

Bootstrapping

Answer 10

a resampling technique that generates samples of size N (called bootstrap samples) from an initial dataset by randomly resampling ( drawing with replacement) N observations.