Part 2. K Nearest neigbors

Question 1

Name examples for using K nearest neigbors

Answer 1

Abnormalities
Email spam/not spam
Classifying credit card

Question 2

What is a training set?

Answer 2

In a training set each record contains a set of attributes, one of the attributes is the class.

Question 3

What is a validation set?

Answer 3

Used to determine the accuracy of the model.

Question 4

What is usually the case with the training and validation set?

Answer 4

Training set is used to build the model and validation set used to validate it.

Question 5

What does a Nearest-Neighbor Classifier require?

Answer 5

Set of stored records
Distance metric to compute distance between records
value of K.

Question 6

How is a unkown record classified?

Answer 6

Compute distance to other training records.
Identify K nearest neighbors
Use class labels of nearest neighbors to determine the class labels of uknown record.

Question 7

What does a small distance imply?

Answer 7

Discriminating attributes are equal and that also

implies that they probably are in the same class.

Question 8

Choosing the value of K

Answer 8

If K is too small, it is sensitive to noise points.

If K is too large, the neighborhood may include points from other classes

Question 9

How to find the weights?

Answer 9

With gradient decent.

Question 10

How to K?

Answer 10

Try many different values of K and look for optimum.

Question 11

What is curse of dimensionality?

Answer 11

In high dimensions (many attributes) everything is far. There are no points nearby. Unless a large number of data is available.