Classification

Question 1

What are the two types of Classification methods, build tree first or create model for each query.

Answer 1

Eager builds the tree first. Creating model takes a while, processing queries doesn’t.
Lazy doesn’t build a tree. Queries take longer. Lazy adapts better to new data.

Question 2

What is entropy for a 50/50 split?
What is entropy for a 25,25,25,25 split (one attribute)?
What is entropy for two 50/50 splits?

Answer 2

50-50: 1.
25,25,25,25: 2
(50/50)(50/50): 1

Question 3

Information Gain and Gain Ratio

Answer 3

Gain: old entropy - new entropy
Ratio: new Entropy / Old Entropy

Question 4

What is Gini for 50/50 split?

Gini approaches 1 as the split gets worse.

Answer 4

50/50: .5

.25,.25.25.25: .75

Question 5

Three ways to turn a numerical value into an ordinal value. (Whats this called?)

Answer 5

Discretizing:

1) Equal Width: select arbitrary range size
2) Equal Depth: Select cluster size
3) Distance-based ???

Question 6

Whats an entropy based Discretization method

Answer 6

Done recursivly. For all possible splits in your data, find the split that makes entropy lowest. Repeat on either side of split…
Do as long as information gain is greater than threshold

Question 7

How do you pre-prune or post-prune a decision tree?

Answer 7

Prepruning: accomplished by an information gain threshold
Post-pruning: Sub-tree replacement.
Sub-tree raising.

Question 8

3 methods for dividing sample and test data

Answer 8

Holdout: divide a dataset into 2/3 training 1/3 test.
Random Sampling: Train on 100% of data, test on random subsets.
Cross Validation. (K-fold subsets)

Question 9

Discribe a method to test your classifier that penalizes mis-predictions

Answer 9

Cost matrix.  Confusion matrix where costs are assigned to each quadrant.
In cancer screen example, 
TP = -1
FP = 1
FN = 100
TN = 0

Question 10

Whats the rule-based eager method?

Answer 10

Sequential Covering Algorithm.

Ponder: “As rules grow, certanty increases, coverage decreases.”

Question 11

What weakness of trees does Sequential Covering alg really beat

Answer 11

Subtree Duplication:

Question 12

Two attributes of a good rule set

Answer 12

Exclusive and Exaustive:
Exclusive: no entry wll match two rules.
Exaustive: any inbound query will hit.

Question 13

What two attributes do you ranke rules on?

Answer 13

Coverage: Fraction of records that satisfy Antecedent.
Accuracy: Fraction of records that satisfy both!
If X, then Y.
Lots of X’s, high coverage. Lots of XY, then good Accuracy

Question 14

Naive Bayes probability summary

Answer 14

For all attributes, you need the percentage of (d1 | X). So of 9 days to play golf, 3 of them were sunny. 1/3.
Multiply all of those times the P(X)/P(d1d2d3X)

Question 15

Remind yourself how to handle zeros in data

Answer 15

Its that shit where you add one of each ordinal…

Question 16

SVM: Support Vector Machine basics

Answer 16

This is the one that trys to find a hyperplane to classify all records on.

Question 17

What are four applications of classification analysis

Answer 17

Explaining Why.  
Credit approval
Target Marketing (likely to buy)
Is medical treatment effective?
Medical Diagnosis

Question 18

What is KNN classification?

Answer 18

K-Nearest Neighbors. I’m betting it is cosine of attribute vectors.

Question 19

List all the lazy and non-lazy methods you can think of.

Answer 19

Lazy: KNN, Range-query (similar to knn). Naive Bayse
Eager: Support vector machine?
Tree Generation
Sequential Covering Algorithm

Question 20

Supervised versus Unsupervised learning

Answer 20

Supervised (like tree generation) Training data has labels or classes.  This is classification.
Unsupervised, class labels are unknown, try to discover classes/labels.  (clustering)

Question 21

What is an advantage of Gain Ratio over Information Gain

Answer 21

Gain ratio tries to penalize larger splits (split on first-name example)

Question 22

What is a class-based discretion process

Answer 22

Place breakpoints in a numeric data.  
1).  Place breakpoints in between numeric values where the class changes.
2).  Set a minimum number of values to have in a class.  Place breakpoints where majority of class changes.
(YNYYY) (NNYYY) (NYYN)