Ensemble Learning Algorithms With Python

Question 1

WHAT ARE THE REASONS WE USE ENSEMBLE ALGORITHMS? P6

Answer 1

Reliability: they can reduce the variance of the predictions
Skill: Ensembles can achieve better performance than a single model

Question 2

WHAT ARE THE CLASSES IN SCIKITLEARN.ENSEMBLE FOR USING BAGGING IN DIFFERENT TYPES OF PROBLEMS? P7

Answer 2

BaggingClassifier and BaggingRegressor

Question 3

DOES RANDOM FOREST USE ALL THE FEATURES FOR MAKING ITS DECISION TREES? EXPLAIN HOW IT OPERATES P9

Answer 3

No, it randomly chooses a subset of features for each tree and considers only them at each split point

Question 4

WHAT IS THE PARAMETER FOR CHOOSING THE NUMBER OF FEATURES FOR RANDOM FOREST TO CONSIDER AT EACH SPLIT CALLED?

Answer 4

max_features parameter

Question 5

WHAT IS THE BASE MODEL FOR ADABOOST? WHAT ARE ITS PROPERTIES? P11

Answer 5

Decision tree (but since they are very short, they’re called decision stumps, they make a single decision on one input variable before making a decision)

Question 6

WHAT IS EXPECTED FROM THE MODEL WHEN WE ADD MORE ENSEMBLE MEMBERS? P11

Answer 6

New models attempt to correct the errors made by prior models already added to the ensemble. As such, the more ensemble members that are added, the fewer errors the ensemble is expected to make

Question 7

WHAT ARE THE TWO MORE EFFICIENT IMPLEMENTATION OF GRADIENT BOOST CALLED? P13

Answer 7

XGBoost and LGBM

Question 8

WHAT IS THE MAIN DIFFERENCE BETWEEN VOTING AND STACKING? P17

Answer 8

In stacking, a final estimator is used to learn how to best combine the predictions of the base models.

Question 9

WHICH MODELS ARE USUALLY USED AS THE FINAL ESTIMATOR IN STACKING ENSEMBLE? P17

Answer 9

Linear models: Linear regression and Logistic regression but can be any model in general