Ensemble Learners

Question 1

Ensemble Learners

Answer 1

Combined group of learning algorithms

Question 2

Ensemble Process

Answer 2

Classification: Have each learner “vote” on answer

Regression: Take mean of result from each learner

Question 3

Ensemble Pros

Answer 3

Lower Error
Less Over fitting

The bias of each learner “fights” (counteracts) bias of others

Question 4

Bootstrap Aggregating (“Bagging”)

Answer 4

Use same training algorithm, but train each learner on different data set

Question 5

Bag Creation: Random with Replacement

Answer 5

Randomly select samples from training set, samples can repeat

Question 6

Boosting

Answer 6

Focus on areas where system is not performing well

Question 7

AdaBoost

Answer 7

increase selection chance of points for next bag that had high error in first bag

Test error on ensemble of previous learners to re-weight picks for next bag

Question 8

AdaBoost Overfit

Answer 8

As # bags increases, AdaBoost assigns more specific data points to learners, leading to more overfit