Lecture 8 Tree

Question 1

Tree

Answer 1

Non- linear
Doesn’t care about scaling of distribution
Interpretable

Question 2

Building decision tress

Answer 2

Individual tree is built on a subset of data

Question 3

Criteria ( classification

Answer 3

Gino index
Cross entropy

Question 4

Regression tree

Answer 4

MAE, MSE, predict mean, without regularization / pruning- each lead often contains a single point to be pure

Question 5

Parameter tuning

Answer 5

Pre- pruning and post

Depth, leaf nodes, samples split

Question 6

Drawback

Answer 6

Extrapolation— only based on current range of the training — nearest leaf node — no ability to generate new response

Question 7

Instability

Answer 7

Split data different may get different root nood, unstable feature importance , may take one or multiple from a group of correlated features

Question 8

Splitting method

Answer 8

Linear models used if extrapolation is needed

Question 9

Ensemble models

Answer 9

Method that combine multiple machine learning method to create more powerful method

Question 10

Poor man’s ensemble

Answer 10

More models —> better if they are not correlated—> average the result

Question 11

Bagging

Answer 11

Generic way to build slightly different models

Question 12

Bias and variance

Answer 12

Generalization depends on strength of individual classifiers and inversely on their correlation

Strength: ability to accurately predict the target variable

High strength— low bias

Uncorrelating—> help