Classification Algorithms

Question 1

What are the steps of generating a decision tree classification?

Answer 1

1. Calculate entropy of root node (or whole dataset if root not exists)
2. Calculate entropy of all other attributes
3. Calculate information gain for each attribute
4. Pick lowest information gain to use as your ‘split’
5. Traverse each decision. If dataset is sorted, close. Otherwise, repeat.

Question 2

What is the entropy equation?

Answer 2

The sum of -pi log(pi) for all classes in that attribute, where pi is the probability that the class appears over each other.

Question 3

What is the information gain equation?

Answer 3

Entropy of the current node minus the sum of the entropy of each class of the chosen attribute multiplied by the probability that class will appear given t.

Question 4

What are the steps of generating a K-nearest neighbours classification?

Answer 4

1. Calculate the distance of each node in respect to the new node.
2. Find its K nearest neighbours, where K is a user input.
3. Find the most common class among those neighbours and select it.

Question 5

What is different between K-nearest neighbours classification and regression?

Answer 5

A K-nearest neighbours regression takes the average of the neighbours’ numerical values. Sometimes average isn’t used, instead opting for max. Either way, we use some measure to find a calculation between the points.