Classification

Question 1

When is classification in modeling useful? (read: used)

Answer 1

When the response (outcome Variable) is nominal or categorial.

Question 2

Must there be an order of the levels in the categorial response in order to use classification?

Answer 2

No, there must be no order in the levels of the categorial response, otherwise other techniques may and should be applied.

Question 3

In classification, what is the vocabulary?

Answer 3

The vocabulary is the set of Classes C that the response value can be devided into. f.i. {yes, no} or {bus, tram, bike, train}

Question 4

What does a classifier do?

Answer 4

It takes as an input the features (IV’s) and predicts what class (C) most likely belongs to categorial resonse Y.

Question 5

What is Multi-Label Classification?

Answer 5

When the instances we test may be associated with more than one class.

f.i. an image containing a house and a dog may be classified as BOTH a house and a dog.

Question 6

What is Hierarchical or Multilevel Classification?

Answer 6

The classes in set C can be divided into subclasses, so it hierarchical.

Question 7

What is Structured Classification?

Answer 7

If there is structure in the input, there must also be a structure in the output.

f.i. each word of a sentence can be classed as {noun, verb, etc} but a sentence has to contain a certain structure to be comprehendable.

Question 8

In Binaray or Multiclass Classification, how do we calculate Accuracy?

Answer 8

Where TP/TN means True Positive/Negative

Question 9

Is accuracy a good metric for classification?

Answer 9

Only if both cases are well represented, this is because of imbalance.

Question 10

In classification, what is imbalance?

Answer 10

If one response appears more frequent than the other responses in our data, there is an natural skew in the classifier towards classifying this response class.

Question 11

In classification, what is recall and how do we calculate it?

Answer 11

fraction of predicted that are correct.

(TP/NF) = true positive/ negative false

Question 12

In classification, what is precision and how do we calculate it?

Answer 12

fraction of predicted that are correct.

(TP/NF) = true positive/ negative false

Question 13

In classification, what is the F-score?

And what does it’s value represent?

Answer 13

The F-score is an harmonic mean to balance Precision and Recall.

A high F-score means both P and R are high as well, generally a good thing.

Question 14

For what types of classification can we use classification as regression?

Answer 14

Only binary classification. However for classification as logistic regression the multi levels can be seperated as well.

Question 15

In classification as regression, what type of regression is useful?

Answer 15

Logisitc Regression, as the sponse space ensures the probabilty of the link space between the binary values of 0 and 1, hence the threshold is always crossed.

Question 16

In tree modelling using regions, explain top down greedy approach.

Answer 16

Top-down: from root to leaves
Greedy: make the best split at each point, without looking back or forward

Question 17

In tree modelling, what is the ideal region accuracy using the Gini Index?

Answer 17

If all instances in a region belong to a certain class, and no other regions contain instances of that class. So the gini index is 1.

Question 18

In tree modelling, what is pruning?

Answer 18

Pruning is the reduction of an existing tree in order to reducing variance and overfitting.

Question 19

What are hyperparameters?

Answer 19

They are not parameters of the model, but of the training process.

Question 20

What process is used to determine how much to prune a tree?

Answer 20

Same as other overfitting models: cross-validation on tree size.

Question 21

in terms of bias and variance, how do the values change when pruning deep trees?

Answer 21

reduce variance but increase bias

Question 22

in terms of bias and variance, what values are expected in deep (read: large) trees?

Answer 22

low bias but high variance

Question 23

What is bagging in modelling and how does it work?

Answer 23

Bagging is a technique to reduce the model complexity without reducing bias (compromising accuracy). often used in tree pruning.

In bragging you create multiple models (trees) with multiple training sets and average their predictions. this reduces variance.

Question 24

Where are the training sets comming from when using bagging?

Answer 24

You sample (or bootstrap) the (sub) training sets from the full training set.

Question 25

Why and how do Random Forests improve the tree model?

Answer 25

Random Forests are used in tree-bagging and reduce variance even more by decorrelating the trees: In normal bagging you use all the predictors in every model, this means most models will be similar. Hence Random Forests make use of only a (random) selection of the predictors in each tree split.

Question 26

What does an Support Vector Machine do?

Answer 26

It finds a hyperplane that separates the classes in the (2D) feature space.

Question 27

In SVM’s, what are margins?

Answer 27

When a hyperplane devides an space into two classes, the margin defines the gap between the classes and the hyperplane

Question 28

What are soft margins in SVM?

Answer 28

It accounts for the fact that a perfect hyper plane cannot always exist due to noise. in soft margins, points further than M(1-e_i) units away are still allowed.

(e_i is error from the (correct) margin edge.)

Question 29

In SVM, what is a budget?

Answer 29

A budget is refering to the total amount of error when using Soft Margins to find you hyper plane. Note that the budget is a hyperparameter!

Question 30

In SVM, what is the support set?

Answer 30

It is the set of points that are located inside the margins of an hyperplane.

These observations determine the position of the hyperplane on their own.

Question 31

Do I have a low or high budget in SVM when my variance is high and my bias is low?

Answer 31

low budget, as few points are contained in the margin so variance is high, and accuracy is high as well hence low bias.

Question 32

Do I have a low or high budget in SVM when my variance is low and my bias is high?

Answer 32

high budget, as many points are contained in the margin so variance is low, but accuracy is low as well hence high bias.

Question 33

In SVM for Multiclass Classification, explain the One versus All approach.

Answer 33

You make a hyperplane for each class of a model and choose the class for which the hyperplane best predicts. (f_k(x) is largest)

Question 34

In SVM for Multiclass Classification, explain the One versus One approach.

Answer 34

You model each possible pair of classes in you predictors and choose the class that wins in most cases.