Classification: Basic Concepts

Question 1

What is classification?

Answer 1

model or classifier is constructed to predict class (categorical) labels

Question 2

What is a numeric prediction?

Answer 2

model constructed predicts a continuous-valued function, or ordered value, as opposed to a class label

Question 3

What is regression analysis?

Answer 3

statistical methodology that is most often used for numeric prediction

Question 4

What are the two major types of prediction problems?

Answer 4

Classification and numeric predictions

Question 5

What are the two steps in data classification?

Answer 5

learning step and classification step

Question 6

what is the learning step

Answer 6

The training phase where classification algorithm builds teh classifier by analyzing or learning form a training set with the associated class labels.

Question 7

What is classification step?

Answer 7

model used to predict class label for a given data

Question 8

What is the accuracy of a cllassifier?

Answer 8

In a given test set, is the percentage of test set tuples that are correctly classified by the classifier.

Question 9

What is desicion tree induction?

Answer 9

learning of decision trees from class-labeled training tuples

Question 10

What is a desicion tree?

Answer 10

flowchart-like tree structure

Question 11

What does each internal node (non leaf node) denotes?

Answer 11

a test on an attribute

Question 12

What does each branch in a desicion tree represent?

Answer 12

outcome of the test

Question 13

What does each leaf node represent?

Answer 13

terminal node holds the class label

Question 14

How are decision trees used for classification?”

Answer 14

Given a tuple, X, for which the associated class label is unknown, the attribute values of the tuple are tested against the decision tree. A path is traced from the root to a leaf node, which holds the class prediction for that tuple

Question 15

Why do we do attribute selection measures?

Answer 15

used to select the attribute that bests partitions the tuples into distinct classes.

Question 16

What is an attribute selection method?

Answer 16

heuristic procedure for selecting the attribute that bests discriminates the given tuple according to class

Question 17

What are two examples of attribute_selection_methods?

Answer 17

ifnormation gain and gini index

Question 18

Does gini index enforce non binary or binary trees?

Answer 18

binary

Question 19

What is the other name ofr attribute selection methods?

Answer 19

splitting rules

Question 20

What is the bias in information gain?

Answer 20

bias towards multivariate attributes

Question 21

What is the bias in gain ratio?

Answer 21

prefers unbalanced splits in which one partition is much smaller than the others

Question 22

What is the gini index bias towards?

Answer 22

bias towards multivariate attributes and it favors tests that result in equal sized partitions and purity in both partitions

Question 23

When does the gini index has difficulti with?

Answer 23

when the # of classes is large

Question 24

What is the best attribute selection method in regards to not being biased for multivariate attributes?

Answer 24

Minimum description length (MDL)

Question 25

what does MDL stand for? What does it do?

Answer 25

Minimum description length uses encoding techniques to define best desicion tree as the one that requires the fewest number of bits to both encode the tree and encode the exceptions to the tree. basically simplest solution is the best.

Question 26

what is a multivariate split?

Answer 26

form of attribute (Feature) construction where new attributes are built based on existing ones

Question 27

What does tree prunning does?

Answer 27

addresses the problem of overfitting the data in desicion trees

Question 28

What types of measures do tree prunning uses?

Answer 28

statitstical measures to remove the least-reliable branches

Question 29

What are the two most common tree prunning approaches?

Answer 29

prepruning and postprunning

Question 30

How does preprunning work?

Answer 30

tree is pruned by halting construction early. upon halting node becomes leaf and the leaf may hold the most frequent class among the subset tuples or the probability distribution of those tuples

Question 31

How does postprunning work?

Answer 31

removes subtrees from fully grown trees. removes branches and replaces it with a leaf which is labeled with the most frequent class among the subtree being replaced

Question 32

What is the cost complexity?

Answer 32

prunning algorithm that considers the cost complexity of a tree to be a function of the number of leaves in the tree and the error rate of the tree

Question 33

What is the error rate?

Answer 33

percentage of tuples misclassified