Week 10 DSE

Question 1

What are the pros and cons of decision trees?

Answer 1

pro:
- no need to standardize data
- perform variable selection
- computationally efficient
- scales up very well, allows interpretation

cons:
- simple trees will likely lose out to other methods on prediction (unlike knn)
- don’t naturally lead to continuous models
- does not handle categorical var well when many categories
- can increase predictive performance but at the cost to interpretability (big, deep trees)

Question 2

What is discontinous data?

Answer 2

• Change data would cause the important varibales predicted to be different
• highly dependent on training data

Question 3

What is root node and leaf node?

Answer 3

• root: first node
• leaf: final decision

Question 4

What is covariate space?

Answer 4

space of x variable

Question 5

Regression tree ______________ the _____________ into a set of rectangles and then fit a _______________in each one.

Answer 5

splits effectively/partition
covariate space
simple model (constant)

Question 6

What ar ethe steps of building a tree?

Answer 6

CART
- form the tree using recursive binary partition of data

Question 7

To make a prediction, just find the ___________to which the new observation belongs and equate the _________to the __________ of that region.

Answer 7

interval
forecast
sample mean

Question 8

What is the differnece between the splits if xi is numeric vs categoric?

Answer 8

• For numeric xi , the rule is based on the threshold xi < c.
• For categorical xi , the rule lists the set of categories sent left.

Question 9

What is used to measure complexity of decision trees?

Answer 9

number of leaves/terminal nodes (T)

Question 10

What is usually the loss funciton of decision trees?

Answer 10

MSE

Question 11

What does alpha stand for in the tree choice minimisation problem

Answer 11

penalty parameter

Question 12

What happens when alpha is 0?

Answer 12

we would choose the tree that perfectly fits the data resulting in overfitting

Question 13

What is the algorithm in choosing tree size?

Answer 13

cost- complexity pruning (weakest link pruning)

Question 14

What are the steps for the tree minimisation problem?

Answer 14

Step 1: Grow big. Use recursive binary split with stop condition.
Rationale: seemingly worthless split high in the tree may really play an important role lower down

Step 2: Prune back. Recursively prune back big tree. Examine every pair of leaves and eliminate if result in SMALLEST increase in loss. Give sequence of subtree of initial big one that contains T hat

Question 15

What does smaller cp mean?

Answer 15

means smaller α
small penalty on complexity and hence bigger tree.

Question 16

_________ has CV- min
cp.

Answer 16

Middle tree

Question 17

Can decision trees help to improve linear regression model?

Answer 17

Yes
The tree can give us ideas about important interactions which can help further improve the linear regression model.

Question 18

Is it possible to use MSE for binary y?

Answer 18

YES

Question 19

Why is misclassification not recommended for use as loss for growing tree ?

Answer 19

Misclassification is not differentiable, reasonably insensitive to node probabilities

Question 20

What is the gini error?

Answer 20

error rate for a rule that assigns each observation to outcome k with probability . In the binary case, it is just the variance at the node.

Question 21

What loss is used for growing tree? What is used for pruning?

Answer 21

Growing tree: gini or deviance

Pruning: Misclassification

Question 22

Which loss is suitable for cost complexty pruning? Which one is the most common?

Answer 22

All

misclassification most common

Question 23

Trees cannot perform variable selection (T/F(

Answer 23

FALSE

Perform variable selection, detect nonlinearities and interactions without careful user specification.

Question 24

What is a notable feature of decision trees?

Answer 24

automatic interaction detection

(detects Interaction between 2 variables)

Question 25

What is T hat alpha, and Q?

Answer 25

T hat alpha: optimal tree structure
Q= argmin t (L(T,y)+alpha|T|

Question 26

when alpha is infinity, what kind of model do you get?

Answer 26

simplest possible model, which is predicting sample mean for any prediction

Question 27

What do you get afte rthe tree minimisation algo?

Answer 27

sequence of subtrees of the initial big one that must contain T hat alpha