Instance Based Learning

Question 1

What is the assumption of Inductive Bias in k-Nearest Neighbours?

Answer 1

Class of instance Xq is the most similar to the class of other instances that are “nearby”

Question 2

What are the components of an IBL algorithm?

Answer 2

Distance function

Classification method

Memory updating

Question 3

What is the distance function?

Answer 3

It returns a measure of the distance between 2 instances

Question 4

What is the classification method component?

Answer 4

1-nearest neighbour (1-NN): the new instance (from test set) is assigned the class of its nearest neighbour (from the training set)

K-nearest neighbour (K-NN): the new instance is assigned the majority class in the K nearest neighbours (K is normally set to 3, 5, etc)

Question 5

What is the memory updating component?

Answer 5

Simplest approach: saves all training examples (but takes lots of memory and processing time)

“intelligent” methods select the most relevant instances

Question 6

What is the motivation for attribute weighting?

Answer 6

Different attributes have different degrees of relevance

Question 7

What are the advantages of IBL?

Answer 7

Simplicity

Suitable for complex problems, with strong attribute interaction

Incremental (new data can be added immediately, don’t need to re-build the model)

Question 8

What are disadvantages of IBL?

Answer 8

Classification of test instances is slow

Question 9

In the context of data mining, what do K-NNs return for IBL?

Answer 9

They return a very specific explanation for the classification. I.e., the nearest neighbour(s) but not a generalised explanation (rule)