5/6 - Unsupervised Learning in the era of generative AI

Question 1

Unsupervised learning

Answer 1

Learning from data without labels

Question 2

Unsupervised v supervised

Answer 2

• Unsupervised: cheap; no definition of error; can discover new things
• Supervised: expensive labelling; require def of error; can do as well as the labels

Question 3

Clustering

Answer 3

Detecting that data points can be grouped into distinct clusters.

Question 4

What metric is required for clustering?

Answer 4

Compute distances

For example euclidean distance or manhatten distance

Euclidean is length of the line between them
Manhatten is the distance if you had to go horizontal then vertical for example.

Question 5

Hierarchical Clustering

Answer 5

1. Each point is a cluster
2. Compute all distances
3. Find shortest distance among any two points and merge clusters.
4. Recompute distance matrix with all points and new cluster
5. repeat from 3.
6. Until you have 1 then looking back you have a tree

Question 6

Clustering Means

Answer 6

K means
Hierarchical

Question 7

K means clustering

Answer 7

1. Start with k known centroids manually chosen
2. “place” each datapoint
3. Compute the new barycentre of each cluster (new centroids)
4. repeat point 2
5. stop when centroids no longer move

Question 8

principal component analysis/dimensionality reduction

Answer 8

Find the direction of maximum variation of the data and this we can reduce the dimensions (eg using an auto encoder) .

Finds the new axis and “rotates” the data to use this line as the x axis.

Question 9

Auto encoders

Answer 9

Neural Net that learns to compress/effectively represent data without labels

This is a neural network with input and output of the same dimensions and a bottleneck in the middle.

x1 x1*
\ /
.
/ \
x2 x2*

. is a latent value (reduced dimensions/compressed layer)

Question 10

How does an auto encoder learn weights?

Answer 10

Backprop: Loss computed as the difference between input and reconstructed output eg L2=1/2(x*-x)^2

FOrward:
- Feed first datapoint to input,
- compute loss as above,
- then propagate gradients w/r to the weights.
-update weights by multiplying them with negative of the gradients (times a learning rate)

Question 11

Can you perfectly capture data again after reducing the dimensionality to 1then back to 2?

Answer 11

No, there will be some variation .

You can produce new data points from the original that is not exactly the same.

Question 12

Generative Adversarial Networks

Generator/Discriminator

Answer 12

Input random noise
network generates fake image

then discriminator network is given real and fake images.

You train generator to create images that produce a real response from the discriminator.

Question 13

GANs: Discriminator outputs

Answer 13

Discriminator outputs likelihood in range 0->1 that image is either fake (0) or real (1).

Generator wants to produce images that are given a 1.

Question 14

Interpolating generator images

Answer 14

You can transition the input vector and sum or subtract values to produce all sorts of interpolations.

Question 15

Conditional GANs

Answer 15

Take additional input to specify which class of objects you want to generate.

With random vector, introduce condition.

Question 16

pix2pix: Adversarial Loss

Answer 16

Ensures the generated images are indistinguishable from real to the discriminator

Question 17

pix2pix: L1/L2 Loss

Answer 17

Ensures images are structureally similar to target images

Penalises pixel-wise differences between generated and real images.

Question 18

CycleGAN

Answer 18

Two generators, two discriminators.

Ga->b translateas domain a to b
Gb->a translates domain b to a

Da differentiates between domain a and translated images.
Db in reverse

Cycle:
A loss penalises when an image converted and converted back does not equal the original image.

Question 19

Hierarchical Clustering: single linkage

Answer 19

the min dist between any two points from the two clusters

Question 20

Hierarchical clustering: Complete Linkage

Answer 20

is the max distance between any two points from the two clusters

Question 21

Hierarchical Clustering: Average Linkage

Answer 21

Average distance between all two points, one from each cluster.