multi-dimensional scaling and Cluster analysis

Question 1

why are we covering both CA and MD techniques in one class?

Answer 1

because they are complimentary to eachother

can merge the two techniques and use them together

Question 2

what is CA and MDS?

Answer 2

Two types of exploratory techniques

• help us to understand and locate structure and relationships in the data
• groups objects together based on their characteristics
• looks for patterns of information

Question 3

whats the difference between factor analysis and cluster analysis/MDS

Answer 3

FA

• starts with individual variables and reduce these into dimensions of factors
• different ways to run factor analysis. -look at the correlation structure and try to reduce it using the factor loadings
• interpret where the dimensions are based on how individual variables load on these actors

cluster analysis/MDS

• start again with individual variables
• then determine which ones go together

difference - we don’t extract dimensions, instead just try to determine which variables in the dataset go together. this is something YOU do. you aren’t presented with extracted factors your’e only presented with patterns of how things might go together and then you decide which go together.

Question 4

in which discipline would you use cluster analysis

Answer 4

Used in almost every discipline: psychology, neuroscience, biology etc.,

Question 5

sometimes we need to sort variables together

Answer 5

the criteria we use to do the sorting will affect the outcome of the sorted variables

Question 6

what is cluster analysis

Answer 6

Humans are good at identifying patterns - e.g., just looking ta the residual plot reveals a pattern

very difficult to identify patterns mathematically

CA provides you with information that you can use to identify what the patterns are. human-machine work together

Question 7

what is a dissimilarity matrix

Answer 7

where the larger the number the more dissimilar our two objects (e.g., the di stance between two cities)

Question 8

what is a similarity matrix and can you give an example of this

Answer 8

where the larger number indicates two objects are more similar e.g., a correlation table

Question 9

what do we need to be aware of when running cluster analysis with regrads to the matrix

Answer 9

whether it is a dissimilarity or similarity matrix

Question 10

what does cluster analysis actually do in terms of points of data

Answer 10

it puts points that are most similar together and pushes points most dissimilar apart

clusters things together

Question 11

what different techniques are used to cluster things together

Answer 11

• k means clustering - non-hierarchical method. you decide in the beginning how many clusters you want. run it then get a suggested membership of data points to clusters

Question 12

what is k means clustering?

Answer 12

a non-hierarchical clustering method

• we pick some starting cluster numbers - e.g., I want 3 clusters
• algorithm starts by randomly picking 3 cluster points in your data set
• at each step - clustering algorithm calculates the distance between each data point and the cluster center and assigns each datapoint membership to the cluster group nearest
• THEN - cluster center is moved by a certain algorithm - calculates whether this improved the distance measure between all data points and the cluster center

so the goal is to do iterative procedure to

• find the cluster center
• having the goal number (e.g., 3)
• and find the position of those cluster centers that will minimise the distance of all data points that could be assigned to that cluster

Question 13

Explain what’s happening in this k means clustering shite

Answer 13

Well a cluster of 3 has been identified, the three points have been shifted 4 times to find the ideal location for data points closest to the cluster

Question 14

with k means clustering what is shifted around the screen - the data points to find those fitting best to the clusters OR the cluster points moving until the data points are closest to it

Answer 14

The data points STAY PUT its the cluster point that shifts bit by bit and stops when the data points for the desired number of clusters are closest

Question 15

with k means clustering, if the cluster centeroid shifts far enough, is it possible for data points to be assigned a different cluster membership

Answer 15

yes

Question 16

when does k means clustering stop

Answer 16

when any further change in the cluster center doesn’t reduce the differences anymore.

Question 17

what does the p-value in cluster analysis tell us

Answer 17

there is no p-value nor end statistic of any sort. you are only presented with, e.g., for. k means clustering, a suggestion of cluster membership for different data points

Question 18

describe non-hierarchial cluster analysis

Answer 18

non-hierarchial methods

• where clusters are formed by assigning membership to clusters
• you decide how many clusters you want before the analysis e.g., k means cluster
• individual data poitns are assigned one of them according to some particular criteria

Question 19

in non hierarchial cluster analysis how might you decide on the number of clusters?

Answer 19

• have a certain theory
• use previous literature - look the number they used
• run it with varying numbers e.g., 2-5 then see which one gives the most reasonable cluster groups

Question 20

hierarchial method for cluster analysis: what are the two groups?

Answer 20

• agglomorative method
• divisive method

in any hierarchical method it goes from 1 to many clusters or many to 1. typically presented either a dendrogram or an icicle plot. then YOU determine themeaningful number of clusters is using a cut off.

in both cases you get a tree diagram (Dendrogram) and an icicle plot -helpful in deciding a feasable cut off point

Question 21

hierarchial cluster analysis: agglomerative methods

Answer 21

different types: single link (nearest neighbour), maximum link (furthest neighbour) or average link (centeroid clustering) - they differin the way they compute the distances

• start by treating each data point as a one-member cluster
• then proceed to put things together - agglomerate clusters
• once a pair of object shave been put together - cant split them up again
• means new clusters are formed based on clusters already created at a previous step

Question 22

hierarchical cluster analysis: devisive methods

Answer 22

• treat all data points as one giant cluster
• then split things up - once a pair has been seperated they can never join again

Question 23

what is the single link aka nearest neighbour technique

Answer 23

one method of hierarchial agglomerative clustering

• start with each city by itself
• then start amalgamating them
• looks at the data finds the points with the closest relationship to each other (durham-subderland) and group these together in a cluster
• then distance matrix recalculated and finds the cities that are next in line closest together (including the Durham-Sunderland cluster as one)

Question 24

single link aka nearest neighbour technique

look at the dendrogram and name are the cluster groups in order

Answer 24

1. durham and sunderland
2. exeter and plymouth
3. birmingham + (Exeter + plymouth)

horizontal axis is a measure. of the relative proximity of the variables e.g., relationship between Durham and sunderland is closer than the relationship between exeter and plymouth. knowing the relative distance between cities can help you to create a cut off point (e.g., cut off point at about 3 in the x axis number line would give. usonly 1 cluster, nut if it was at 24 we would have 3

Question 25

single link aka nearest neighbour technique

so after Durham and Sunderland form 1 cluster, SPSS recalibrates and computes a NEW dissimilarity matrix. how does it do this?

when we compare other cities e.g., Exeter to these two cities which distance is used in the dissimilarity matrix.

Answer 25

whichever gives us the smaller value in this case the Durham-Exeter distance. Exeter is the closest link.

This is the matrix used to make the second clustering decision - and we see the smallest value in this table is the exeter-plymouth link

Question 26

single link aka nearest neighbour technique

lets say in our dissimilarity matrix were comparing the distance between a cluster of 2 cities anda nother cluster of 2 cities - how do we decde the distance value to use in the dissimilarity matrix?

Answer 26

(A, B) vs (1,2)

compare a with 1 and 2, B with 1 and 2, compare 1 with a and b, compare. 2 with a and b

whichever of these gives us the smalles value we use that in the matrix

keep giong until last 2 clusters form 1

Question 27

Single link aka nearest neighbour technique

looking at the dendrogram how do we now decide how many clusters to have

Answer 27

is it 2 or 3? its a judgment call YOU make the decision

Question 28

Maximum link (furthest neighbour)

Answer 28

• again, durham and sunderland will be the first cluster - bc distance was smallest (19)
• but then the distance computed between this cluster and the other cities // between clusters will use the largest distance
• the SMALLEST value in the matrix is still used to determine the next cluster

Question 29

Average link (Centroid clustering)

Answer 29

Still assigns the two points the smallest distance from on another together – but distances within the table are based on the average between the objects in the table.

Question 30

What’s the difference between single and maximum link hierarchical agglomerative method?

Answer 30

Single link method tends to produce more “chaining” while the maximum link method creates several tightly defined clusters

Question 31

combiing objects and using this as a value in the matrix. why is this bad

Answer 31

SCALE EFFECTS!!!

because the distance matrix is based on the combined scores? whichever variable is bigger (e.g., percent on a maths test > height in meters) will dominate

So if you were going to run a distance matrix you would have to account for this scalling issue

Question 32

What are some scaling issues

Answer 32

• you need to account for any scaling issues when comparing the distance between objects in a dissimilarity matrix
• when similar data are rescaled – e.g., scores on a test – one out of 50 and another out of 75. The raw scores might join child a and b but the percentage scores join child b and c into one cluster

all this is because we use the Euclidean distanceas a measure of proximity. When we combine scores/rescale scores this measure does not maintain the rank ordering you might have in each variable.

Question 33

scaling effects - problem with euclidean distance it doesn’t maintain the rank order

how can we fix this?

Answer 33

• rescale data – z transformation so you basically rescale all scores so they have a mean of 0 and SD of 1. Buts all variables on the same scale so when combining them theyre all weighed equally.
• In SPSS different ways to rescale data – basically just try to put everthing on the same scale
• Which way to chose – depends on your data, if all of them are equally important then z transformation is the way to go
• But if the raw data is meaningful by itself and its not so important he rank ordering is maintained then you might not want to do any transformation

Question 34

With what type of data is it meaningful to compute Euclidean distance. With what is it not?

Answer 34

Interval/ratio scale data. Wouldn’t be meaningful with binary data

Question 35

How can we do a cluster analysis on binary data

Answer 35

Transform the counts into some measure – now can be subject to clustering

• In SPSS different way to re-jigg this data (a,b,c,d) to get a measure of the similarity between x and y
• Measures differ in the way it thinks the absence of a feature is more important than the presence of a feature vice versa

Question 36

Different ways to judge the similarity between 2 binary variables

Answer 36

Simple matching similarity measure (poss most common) – a + d / a + b + c + d

• Basically it’s the total number of matches divided by the total number of measures

Jaccard similarity measure or similarity ratio – a / a+ b + c

• Basically same as SMS but with the double negatives removed

Phi

• Binary form of the pearsons product-moment coefficient

Question 37

After looking at the dendrogram of these birds how can we decide how to cluster them

Answer 37

Go back to your data – ask questions

• Are they woodland/farmland animals
• Are they going up or down in abundance
• Is any species particularly different from the rest

Dendogram doesn’t answer these qs!! Just gives you close where in your data you should look

Question 38

What do you need to be aware of when conducting cluster analysis

Answer 38

• Similarity/dissimilarity matrix - do larger values mean data is more or less similar?
• What decisions to make about the criteria used to cluster the objects (mierarchial max link)
• Type of data you have (interval/ratio/binary) i.e., if you have binary data you may have to transform that into another measure first like simple matching similarity/phi
• Different techniques provide different solutions!

Question 39

What is Euclidean matrix

Answer 39

Dissimilarity matrix