B08 Decision Trees

Question 1

Decision trees are part of a group of learners that use all
the data available to them on a first-come first-served
basis. As a result of this, they are known as __________

Answer 1

greedy

learners.

Question 2

Decision Trees:

-Begins at ____ node.
-Passed through
_____ nodes.
-Data is split across
______ (outcomes).
-Ends at ______ (final decision).

Answer 2

Root Node
Decision Nodes
Branches
leaf/terminal node

Question 3

Decision trees are built using a _______________
approach which splits data into subsets and then
recursively splits the subsets into even smaller subsets
until one or more stopping criteria are met. This
approach is also known as ___________.

Answer 3

recursive partitioning

divide and conquer

Question 4

Some of the criteria that trigger a stop to the recursive
partitioning process include when:
-All data in a leaf node are of the ______.
-All _______ have been exhausted.
-A specified _________ has been met.

Answer 4

same class
features
tree size limit

Question 5

For most decision tree algorithms, the decision about
which feature to split upon is usually made based on a
measure of impurity known as _______

Answer 5

entropy.

Question 6

For decision trees, entropy is a quantification of the
_______________ within a set of class
values.

Answer 6

level of randomness or disorder

Question 7

Entropy is highest when the split is \_\_\_\_\_. As one class
dominates the other, entropy reduces to \_\_\_\_\_\_.

Answer 7

50-50

zero

Question 8

To determine the optimal feature to split upon, the
decision tree algorithm calculates the change in
entropy that would result from a split on each possible
feature. This measure is known as _____________.

Answer 8

Information Gain (F)

Question 9

________ is a modification of information gain that
reduces its bias on highly branching features by taking
into account the number and size of branches when
choosing a feature. It does this by normalizing information gain by the ____________ of a split.

Answer 9

Gain Ratio

Intrinsic information

Question 10

Instead of Entropy, some decision tree algorithms use the __________ to determine the optimal feature to split upon. _______ is a measure of statistical dispersion

Answer 10

Gini impurity measure

Gini

Question 11

Gini impurity goes from ___ to ____ (for an infinite number of
even partitions).

Answer 11

0 to 1

Question 12

A split occurs at the _______ value for the Gini impurity

Answer 12

lowest

Question 13

In R, the __________ uses entropy as a measure of

impurity, while the __________ uses Gini

Answer 13

C50

CART algorithm

Question 14

Decision trees have a
tendency to _____ against
the training data.

Answer 14

overfit

Question 15

To remediate this, the size of
the tree is reduced in order
for it to generalize better. This is known as ____

Answer 15

Pruning

Question 16

The __________________ is used to control the
size of the decision tree and to select the optimal tree
size.

Answer 16

complexity parameter (cp)

Question 17

When used for pre-pruning, if the cost of adding
another variable to the decision tree from the current
node is _______the value of the Complexity Parameter, then tree building does
not continue.

Answer 17

above

Question 18

For post-pruning, the cp
value that corresponds to
the \_\_\_\_\_\_\_\_\_ is used as the
threshold for pruning the
tree.

Answer 18

lowest cross-validation

error

Question 19

Strengths of Decision Trees?

Answer 19

-Does well on most problems.
-Handles numeric and nominal
features well.
-Does well with missing data.
-Ignores unimportant features.
-Useful for both large and small
datasets.
-Output is easy to understand.
-Efficient and low cost model.

Question 20

Weaknesses of Decision Trees?

Answer 20

-Splits biased towards features
with a large number of levels.
-Easy to overfit or underfit.
-Reliance on axis-parallel splits
is limiting.
-Small changes in data result in
large changes to decision logic.
-Large trees can be difficult to
interpret or understand.