Decision Trees

Question 1

What are categorial and what are numeric attributes?

Answer 1

numeric (e.g., number of legs) and categorical (e.g., delicious/not delicious)

Question 2

classification trees and

regression trees difference?

Answer 2

cl: which produce categorical outputs

produce numeric outputs

Question 3

What is entropy?

Answer 3

Entropy represents the uncertainty of the data.
If all the data points belong to a single class, then there is no real uncertainty=low entropy. If the data points are evenly spread across the classes, there is a lot of uncertainty=high entropy

Question 4

What do we have to take care of regarding entropy if we partition data?

Answer 4

partition needs low entropy if it splits the data into subsets that themselves have low entropy (i.e., are highly certain), and high entropy if it contains subsets that (are large and) have high entropy (i.e., are highly uncertain)

Question 5

What do decision nodes and leaf nodes do?

Answer 5

decision nodes lead us through the tree and leaf nodes are the ends which give us predictions.

Question 6

What can random forests do?

Answer 6

They can aviod overfitting by building multiple decision trees and let them decide how to classify.