Decision Trees Flashcards
What does CART stand for?
Classification & Regression Trees
What does a CART model do?
Classify or predict an outcome based on:
a set of predictor variables producing a tree structure
How does a classification tree split?
Splits maximize homogeneity of classes in each branch
How does a regression tree split?
Splits minimize variance within each branch
What are the 4 key features of decision trees?
- Binary splits
- Pruning
- Non-parametric
- Interpretability
What is binary splits?
Each split produces 2 child nodes
What is pruning?
Prevents tree from being grown to minimze complexity & reduce overfitting
What is non-parametric?
No assumapitons about the underlying distribution of each variable
What is interpretability?
Easy to understand visualize
How is a tree produced?
Recursive partitioning
What is recursive partitioning?
Repeatedly split the records into 2 parts
In recursive partitioning, why does a split need to have into two parts?
To achieve maximum homogeneity of outcome within each new part
What are the two ways to measure impurity?
- Gini Index
- Entropy
What are two ways to avoid overfitting?
- Pre pruning
- Post pruning
What are the 5 ways to avoid overfitting when pre-pruning?
- Limit max depth
- Min samples for split
- Min samples per leaf
- Threshold for splitting
- Restrict feature usage
What are the 2 ways to avoid overfitting when post-pruning?
Cost complexity pruning
Cross validation
What are the 5 pros of decision trees?
- Easy to understand
- Handle numerical & categorical
- No need for normalization
- Handle categorical (binary and multi class)
- Not a black box model
What are the 2 cons of decision trees?
- Easy to be too complex
2.Unstable to small variations
