Unsupervised learning

Question 1

For what tasks can we use unsupervised learning?

Answer 1

Dimensionality reduction
Anomaly detection
Visualization

Question 2

What are some challenges of k-means clustering?

Answer 2

Clusters tend to be the same size
Depends on initialization
Handles anisotropic data and non-linearites poorly

Question 3

How can we improve k-means for non-linear datasets?

Answer 3

Change the cost function from euclidean to geodesic/ graph based/ kernel based

Question 4

What is the elbow method?

Answer 4

A methode for determining the number of clusters. (Stop increasing number of clusters when the gain of increase is small).

Question 5

What are the steps of PCA?

Answer 5

1. Create design matrix
2. Center data
3. UDV^T = SVD(1/(N-1) XX^T)
4. Keep the n eigenvectors (columns of U) with largest eigenvalues.
   v* = U^T x
   x = Uv*

Question 6

Describe the steps in AAM (Active appearance models)

Answer 6

1. Calculate the shape model ( mean + eigenmodes troughout the dataset)
2. Warp the image so it fits the landmark template
3. Create appearance model using PCA on the “shape free paches”
4. PCA jointly on the shape and appearance to capture correlations.

Question 7

What are eigenfaces?

Answer 7

“Eigenvectors” created from face images using PCA. These eigenfaces can be used as a basis to reconstruct face images.

Question 8

What are some interpretations of the PCA?

Answer 8

1. Best L2 recontruction error among all linear models of equal rank
2. k’th eigenvector is the direction of maximal variance orthogonal to all former eigenvectors.
3. PCA fits an ellipsoid to the data.

Question 9

How can we interpret PCA probabilistic?

Answer 9

x = Uw + mu + e
w ~ N(0,I)
e ~ N(0, sigma^2*I)

Question 10

How can we enforce sparsity on the PCA?

Answer 10

Add a sparsity weight penalty (L0, L1…)

Question 11

What is one of the main advantages of sparse penalites in the PCA setting?

Answer 11

Can remove noise

Question 12

What is the main dissadvantage of Autoencoders compared to PCA?

Answer 12

1. Visualizing the latent space

2. Generating new samples

Question 13

How can AE be used for image anomaly detection?

Answer 13

Train the autoencoder on “normal” data. When using the autoencoder on a new image subtract the result from the original to find anomalies.

Question 14

What is the difference between GMM and fully Bayesian GMM?

Answer 14

In fully bayesian GMM we assign hyperpriors to pi_n and my_k, sigma_k. A Dirchlete prior on pi_n usually favors fewer non-zero clusters, hyperpriors on mu, sigma can avoid clusters that degenerate to 0 variance.

Question 15

What is variational Bayesian inference?

Answer 15

When we cannot compute the posterior p(theta | X) analytically, we can approximate it using q(theta) from a family of distributions Q.

Question 16

What kind of families Q can we use?

Answer 16

1. Delta dirac ( finds the mode)
2. Parametric familiy of e.g. gaussians
3. Mean field approximation (q factroizes).

Question 17

In variational inference what metric do we use to calculate the distance between two distributions?

Answer 17

KL(q || p) = Ep[log(q/p)]

Question 18

What can we do instead of minimizing KL?

Answer 18

Maximizing ELBO = Eq[log(p(X|theta)] - KL(q||p)