Cluster Analysis

Question 1

Is cluster analysis a supervised or an unsupervised learning method?

Answer 1

Unsupervised learning method

Question 2

Contrast cluster analysis with PCA

Answer 2

PCA: looks for a low dimensional representation that explains most of the variance
Cluster Analysis: tries to group the observations into a small number of groups of similar observations

Question 3

Give examples of application of cluster analysis

Answer 3

. Marketing
. Medical
. Actuarial modeling

Question 4

Give two clustering methods

Answer 4

. K-means

. Hierarchical

Question 5

Define K-means clustering

Answer 5

Group the observations in a data set into K disjoint clusters in which the observations are relatively homogenous.
We decide the number of cluster in advance, they are exhaustive (every observation belongs to one cluster) and mutually exclusive.
The clusters are selected to minimize the total dissimilarities between points within the cluster

Question 6

What is the centroid of a cluster and how can you compute it?

Answer 6

Point whose coordinates are the means of the coordinates of the cluster
Formula: ( (sum(xi)/n ; sum(yi)/n )

Question 7

What is the algorithm for K-means clustering?

Answer 7

1. Split the observations arbitrarily into K clusters
2. Calculate the centroids for each cluster
3. Create new clusters by associating each points with the nearest centroid
4. Repeat steps 2 and 3 until the clusters don’t change

Question 8

What is the dissimilarity function for k-means clustering?

Answer 8

Twice the Euclidean distance of points from the centroid.

Assigning points to the closest centroid can only reduce the total dissimilarity.

Question 9

Define hierarchical clustering

Answer 9

Consists of a series of fusions of observations results in bigger clusters containing smaller clusters containing smaller clusters.
Does not require to specify the number of clusters in advance

Question 10

Describe bottom-up / agglomerative clustering

Answer 10

The algorithm starts with n clusters (1 for each observation) and iteratively fuses the two most similar clusters together until all points are in one cluster.

Question 11

How do you decide how many clusters should be used in bottom-up clustering?

Answer 11

A dendrogram is produced. The number of clusters is determined by deciding at what height to cut the graph.

Question 12

What is a dissimilarity measure?

Answer 12

A formula that measures how different two points are.

Question 13

What is the most common dissimilarity measure?

Answer 13

Euclidean distance (square root of the sum of square differences between coordinates)

Question 14

In what type of cluster analysis do we square the Euclidean distance?

Answer 14

K-means only

Question 15

Why can’t we square the Euclidean distance

Answer 15

in hierarchical clustering, a dendrogram in which the height of each fusion is the dissimilarity measure. We don’t want to spoil the scale of the graph by squaring the distance.

Question 16

How does the fusion algorithm work?

Answer 16

At each iteration, we compare every paire of clusters. If there is K clusters, we male k(k-1)/2 comparaisons.
We select the paire of clusters with the smallest dissimilarity measure and fuse them

Question 17

A dissimilarity between two points is defined by the Euclidean distance (or the square of the Euclidean distance). How is the dissimilarity between two cluster defined?

Answer 17

The dissimilarity between two clusters is called linkage.

Question 18

What are the different type of linkage?

Answer 18

1. Complete
2. Single
3. Average
4. Centroid

Question 19

Define complete linkage

Answer 19

Calculate the dissimilarity between every points of cluster A and every points of cluster B (is there are a points in cluster A and b points in cluster b, do axb calculations). The dissimilarity is the maximum of these numbers.

Question 20

Define single linkage

Answer 20

Calculate the dissimilarity between every points of cluster A and every points of cluster B (is there are a points in cluster A and b points in cluster b, do axb calculations). The dissimilarity is the minimum of these numbers.
This linkage leads to trailing clusters (clusters in which one point at a time is fused to a single cluster)

Question 21

Define average linkage

Answer 21

Calculate the dissimilarity between every points of cluster A and every points of cluster B (is there are a points in cluster A and b points in cluster b, do axb calculations). The dissimilarity is the average of these numbers.

Question 22

Define centroid linkage

Answer 22

Calculate the centroid of each clusters and use the dissimilarity between the centroids.

Question 23

What is the disadvantage of centroid linkage?

Answer 23

Can lead to inversion : a later fusion occurs at a height lower than an earlier fusion - the dissimilarity of a later fusion is lower than the dissimilarity of an earlier fusion involving the same points.

Question 24

True or false : the first link is the same regardless of linkage

Answer 24

True

Question 25

Why does complete linkage usually prefer to fuse smaller cluster together first?

Answer 25

The more points, the higher the maximum tends to be and the fusion is based on the minimum of the measure used.

Question 26

Why does single linkage usually prefer to fuse groups with greater number of observations first?

Answer 26

Opposite to complete linkage: the more observation there is in a cluster, the lower the minimum tends to be.

Question 27

Dose average linkage have a bias towards linking smaller of bigger groups first?

Answer 27

No

Question 28

What is the issue in hierarchical clustering?

Answer 28

It assumes that there is a hierarchy in the variables. If they are unrelated, this may not be a valid assumption. K-means clustering will be superior in those cases.

Question 29

What is another dissimilarity measure that can be used in hierarchical clustering?

Answer 29

Correlation

Question 30

Why does it take at least 3 features to use correlation as a dissimilarity measure?

Answer 30

Any 2 numbers are perfectly correlated with any other 2 numbers.

Question 31

Scale of variable is important. What action can be taken to ensure scale is considered?

Answer 31

One may standardize all variables to have standard deviation of 1

Question 32

Is clustering robust?

Answer 32

No. A small deviation in data can result in different clusters.

Question 33

True or false: one may perform clustering analysis on PCA score vectors instead of on the original data.

Answer 33

True