AdaBoost

Question 1

What is AdaBoost?

Answer 1

AdaBoost also called Adaptive Boosting is a technique in Machine Learning used as an Ensemble Method.

Question 2

What is the most common algorithm used with AdaBoost ?

Answer 2

Decision trees with one level that means with Decision trees with only 1 split. These trees are also called Decision Stumps.

Question 3

How do Decision Stumps work?

Answer 3

What this algorithm does is that it builds a model and gives equal weights to all the data points. It then assigns higher weights to points that are wrongly classified. Now all the points which have higher weights are given more importance in the next model. It will keep training models until and unless a lowe error is received.

Question 4

What happens when the classifier predicts half right and half wrong ?

Answer 4

Then the Total Error = 0.5 and the importance (amount of say) of the classifier will be 0.

Question 5

The amount of say (alpha) will be negative when ?

Answer 5

the sample is correctly classified.

Question 6

The amount of say (alpha) will be positive when?

Answer 6

the sample is miss-classified.