Decision Tree

Question 1

Entropy

Answer 1

Measure of disorder that can be applied to set.
Disorder corresponds to how mixed(impure) the segment is WRT these properties of interest.

Question 2

Entropy formula

Answer 2

-p1 x log(p1) - p2 log(p2) - ..

Question 3

Information Gain measures

Answer 3

measures the change in entropy due to any new information being added.

Question 4

Information Gain formula

Answer 4

– IG(parent,children)=entropy(parent)–[p(c1)×entropy(c1)+p(c2)× entropy(c2) + ⋯+ p(ck) × entropy(ck)]

Question 5

Classification trees

Answer 5

Each interior node in the tree contains a test of an attribute, with each branch from the node representing a distinct value of the attribute.

Question 6

Decision tree - Basic algorithm (a greedy algorithm)

Answer 6

Test attributes are selected on the basis of a heuristic or
statistical measure (e.g., information gain)

Question 7

When do decision tree stop?

Answer 7

There are no remaining attributes for further partitioning.
All sample for a given node belong to the same class.

Question 8

Information Gain drawback

Answer 8

biased towards multivalued attributes

Question 9

Gain ratio drawback

Answer 9

tends to prefer unbalanced splits in which one partition is much smaller than the others

Question 10

Gini Index

Answer 10

• biased to multivalued attributes
• has difficulty when # of classes is large
• tends to favor tests that result in equal‐sized partitions and purity in both partitions