Ensemble Learning

Question 1

Ensemble Learning

Answer 1

Aggregating a group of predictors, you will often get better predictions than with the best individual predictor.

Question 2

2 Way to Constructing Ensemble Method

Answer 2

By manipulating training set - Example: bagging, boosting, random forests
By manipulating input features - Example: random forests

Question 3

4 Approach to Manipulating Training Set

Answer 3

Bagging - sampling with replacement
Pasting - sampling without replacement
Random Forests
Boosting: An iterative procedure to adaptively change distribution of training data by focusing more on previously misclassified records

Question 4

3 Term of Bias-Variance Decomposition

Answer 4

Variance - The distance intervals of predicted value
Noise
Bias - The distance between predicted value and actual value.

Question 5

Bias-Variance Trade-off

Answer 5

Overfitting - Low Bias & High Variance
Optimum - Low Bias & Low Variance
Underfitting - High Bias & Low Variance

Question 6

2 Type of Voting Classifiers

Answer 6

Hard Voting Classifiers (Majority-Vote)
Soft Voting Classifiers - Predict the class with the highest class probability, averaged over all the individual classifiers.