KNN

Question 1

k-NN process overview

Answer 1

Question 2

Goal

Answer 2

given a set of labeled items, automatically label a new item

Question 3

Idea

Answer 3

Consider most similar other items (defined in terms of their attributes), look at their labels and give the unassigned item the majority votes. Ties broken randomly.

Question 4

To automate knn, what two decisions need to be made

Answer 4

• How to define similarity?
• How many should vote? (what is k?)

Question 5

Euclidean distance

Answer 5

Question 6

Cosine similarity

Answer 6

Question 7

Jaccard distance

Answer 7

Question 8

Hamming distance

Answer 8

Question 9

Manhatan distance

Answer 9

Question 10

Regarding distance metrics…what if attributes are a mixture of kinds of data?

Answer 10

Define your own custom designed metric

Question 11

synonymous terms

Answer 11

Question 12

Evaluation metrics

Answer 12

• Accuracy
• Precision
• Recall
• F-score

Question 13

Evaluation Metric : Accuracy

Answer 13

number of correct labels / (total number of labels)

Question 14

Evaluation Metric : Precision

Answer 14

number of true positives /

(number of true positives + number of false positives)

Question 15

Evaluation Metric : Recall

Answer 15

Number of true positives /

(number of true positives + number of false negatives)

Question 16

Evaluation Metric : F-score

Answer 16

Harmonic mean of precision and recall

(2 × precision × recall) / (precision + recall)

Question 17

Evaluation Metric : Misclassification rate

Answer 17

1-accurary

Question 18

Choosing k

Answer 18

• Need to understand data well to get a good guess
• Then try a few different k’s and see how evaluation changes. Pick the k that optimizes the chosen evaluation metric
• In binary classification, pick k to be an odd number

Question 19

Modeling assumptions in K-NN

Answer 19

Question 20

Scaling

Answer 20

Standardize the data so that all variables are given a mean of zero and a standard deviation of one.

In R, this can be achieved using the scale() function