Chapter 9 Quiz Flashcards
splitting node name
decision node
end node name
terminal node
drop new observation down until terminal nodes, assign its class by taking a vote/average of all the training data that belonged to the terminal nodes when the tree was grown
decision tree
dividing up the p-dimensional space of the X variables into non-overlapping multidimensional rectangles
recursive partitioning
what should each rectangle in recursive partitioning be?
as homogeneous/pure as possible
impurity reduction
sum of impurities before split minus sum of impurities for resulting rectangles
stops tree growth before it starts overfiting, assessed whether splitting a node improves the purity in a statistically significant amount
CHAID
uses validation data to prune tree created by training data
CART
tree that minimizes the misclassification error rate of the validation set
minimum error tree
smallest tree in pruning sequence with error within one standard error of the minimum error tree
best-pruned tree
how are classification rules set up
IF…AND…THEN
random forest
fit trees to samples, combine individual predictions and take vote/average
boosted trees
each new tree concentrates on misclassification records from the previous tree
what are the two measures of impurity?
gini measure
entropy measure
gini measure
1 minus sum of observations in rectangle a that belong to call k squared
if owner is 50% and non-owner is 50%, what is the gini measure
1- (0.5^2 + 0.5^2) = 0.50
if owner is 0% and non-owner is 100%, what is the gini measure?
0
entropy measure
log of gini measure
can handle missing values, does not need to standardize
benefits of decision trees
structure is unstable (depends too much on training), can overfit, does not look at correlations, needs large dataset to construct good classifier
negatives of decision trees
what kind of model is decision tree?
clear box, nonlinear, nonparametric
bootstrap aggregating, drawing random samples with replacement (subsets of rows and columns)
bagging
each tree made independent of the one before it
boosting
what types of techniques are bagging and boosting?
perturb (make different models) and combine (create a prediction)
