Lecture 5 - Distance based models

Question 1

What are distance based algorithms

Answer 1

Distance based algorithms are machine learning algorithms that classify instances by computing distances between these instances and a number of internally stored exemplars.

Question 2

____ that are closest to the instance have the largest influence on the classification assignment to the instance

Answer 2

Exemplars

Question 3

What is hamming distance

Answer 3

Hamming distance between two strings or vectors of equal length is the number of positions at which the corresponding symbols are different.
110 and 101 hamming distance 2.

Question 4

What is 0-norm,1-norm and 2-norm give examples.

Answer 4

0. Hamming distance
1. Manhattan distance
2. Euclidean distance

Question 5

What is the Chebyshev distance

Answer 5

The Chebyshev distance, also known as the maximum or L∞ distance, between two points in a space is the greatest of their differences along any coordinate dimension.

Question 6

what are the 4 distance metric conditions

Answer 6

1. Distances between a point and itself are 0
2. All other distances are larger than zero if x!=y.
3. Distances are symmetric
4. Detours can not shorten the distance (tringle inequality)

Question 7

What are exemplars

Answer 7

Exemplars are prototypical instances within clusters/classes

Question 8

What are the two exemplars

Answer 8

1. Centroid
2. Medoid

Question 9

Which one happens in data and which one doesnt

Answer 9

Centroids do not happen in data
Medoids happen in data, more time consuming to calculate

Question 10

Since the number of classes is typically much lower than the number of exemplars, decision rules often take more than one nearest exemplar into account (or k-nearest exemplar)

Answer 10

true

Question 11

What is the curse of dimentionality

Answer 11

In high-dimentional spaces everything is far away from everything and so pairwise distances are uninformative

Question 12

list nearest-neighbour classifier properties

Answer 12

1. Nearly perfect seperation of classes on the training set
2. Easily adapted to real-valued targets and structured objects
3. Unbalanced complexity
4. Perfect seperates training data
   - by increasing the number of neighbours k we would increase bias and decrease variance

Question 13

what are 2 types of distance-based clustering

Answer 13

Predictive clustering
Descriptive clustering

Question 14

What is predictive clustering

Answer 14

use a distance metric, a way to construct exemplars and a distance-based decision rule to create clusters that are compact with respect to the distance metric

Question 15

what is descriptive clustering

Answer 15

trees called dendrograms purely defined in terms of a distance measure are used. they partition the given data rather than the entire instance space.

Question 16

What is k-means clustering problem

Answer 16

The k-means clustering problem is to find a partition that minimises the total within-cluster scatter

Question 17

what are weak points

Answer 17

The algorithm can be impacted by the starting points, you also need to know the number of cluster in advance

Question 18

what is the complexity of medoid

Answer 18

Finding a medoid requires us to calutate for each data point the total distance to all other data points, in order to chose the point that minimises it. Regardless of the distance metric used, this is an O(n^2) operation for n points.

Question 19

how can we evaluate centroids

Answer 19

Inertia: the k-means algorithm aims to choose centroids that minimise the inertia or within-cluster scatter.
Inertia shows how internally coherent and compact clusters are

Question 20

what is a sillhouette

Answer 20

a silhouette then sorts and plots s(x) for each instance, grouped by cluster.
We want high value for b and low value for a

Question 21

What is hierarchial clustering

Answer 21

The k-means algorithms is flat. Hierarchical clustering provides a taxonomy of instances within the cluster, or where clusters can be merged with each other.

Question 22

what is a dendrogram

Answer 22

Given data set D, a dendrogram is a binary tree with the elements of D at its leaves. An internal node of the tree represents the subset of elements in the leaves of the subtree rooted at that node. The level of a node is the ditsance between the two clusters represented by the children of the node. leaves have level 0.

Question 23

What is a linkage function

Answer 23

A linkage function calculates the distance between arbitary subsets of the instance space, given a distance metric.

Question 24

What are the 4 linkage types

Answer 24

Single linkage
Complete linkage
Average linkage
Centroid linkage

Question 25

What are the best 3 linkage in order

Answer 25

1. Complete linkage
2. Centroid linkage
3. Single linkage