2. Decision Tree

Question 1

What is the two different methods of decision tree?

Answer 1

Classification and regression trees

Question 2

Which type of learning can you use with the methods?

Answer 2

Both methods can take supervised and un-supervised learning

Question 3

When do you use classification trees?

Answer 3

When target variable is categorical, splitting based on similar categories and final prediction is a class label

Question 4

How do you evaluate a classification tree?

Answer 4

Classification trees are evaluated through purity and impurity measures

Question 5

When do you use regression tree?

Answer 5

Regression tree is used when the target variable is continous, splitting data into segments of similar value and final prediction being numerical value

Question 6

How do you evaluate regression model?

Answer 6

A regression model is evaluated through computing average loss per observation (either L1, L2) or through MSE

Question 7

What output does classification produce?

Answer 7

Classification tree produce discrete output, learning a function that maps a data object to a discrete class

Question 8

What does classification tree classify?

Answer 8

The relation between attributes and class

Question 9

What can a classifier help predict?

Answer 9

It can help predict the class of a new data object

Question 10

How can we explain the classification tree?

Answer 10

It is represented by a rooted tree, where each node represents a partition of the input space

Question 11

What does branches and leafs represent in classification tree?

Answer 11

Branches = attributes
Leafs = decisions

Question 12

What sentence can explain classification tree?

Answer 12

Ask a series of questions until a conclusion is reached

Question 13

What is the foundation of the impurity measure I(r) or I(vk)?

Answer 13

Impurity measure is ensuring the best split for each split for each node. How impure is a set of data points is with respect to their class labels.

Question 14

What is a good impurity score for classification problem?

Answer 14

Zero is all data points belong to a single class
High values of impurity indicate even mix of classes and is bad

Question 15

In classification what is the three different impurity measures?

Answer 15

Gini index, entropy and misclassification error

Question 16

In classification, when do use each of the three different measures?

Answer 16

Gini index - focus on higher performance (accuracy)
Entropy - information gain, theoretical measure less sensitive to small changes in probability
ClassError - focus on fraction of misclassified instances, incorrect predictions

Question 17

In classification which method is used to see how good the model is?

Answer 17

Purity gain

Question 18

In classification how does the purity gain work?

Answer 18

Also called impurity reduction, evaluating how well a particular feature or attribute splits the data in a decision tree

Question 19

In regression, which method is used to evaluate the model?

Answer 19

Absolute error, average loss per observation

Question 20

In regression, there is two different error calculations?

Answer 20

Mean absolute error (MAE)
Mean squared error (MSE)

Question 21

In regression when should you use MAE and MSE?

Answer 21

MAE - penalizes all errors linearly and is more robust to outliers, used when data is more alike
MSE - penalizes larger errors more heavily, used for larger deviations in the data

Question 22

What different Euclidean loss are there and where are they used?

Answer 22

L1 - MAE, less sensitive to outliers
L2 - MSE, average of the squared difference between predicted values and the actual
L ∞ norm - measure the maximum error, focus on worst case error