Lecture 13 - Unsupervised Learning

Question 1

What is unsupervised learning?

Answer 1

Unsupervised learning involves discovering patterns within data without labeled examples, focusing on describing variability in the data and estimating distributions.

Question 2

What is generative modeling in the context of unsupervised learning?

Answer 2

Generative modeling aims to approximate high-dimensional distributions of data, allowing for the generation of new data samples that resemble the original dataset.

Question 3

Explain unsupervised representation learning.

Answer 3

Unsupervised representation learning involves learning useful weights and representations from unlabeled data, which can be transferred to other tasks with minimal labeled examples.

Question 4

What is the main idea behind self-supervised learning?

Answer 4

Self-supervised learning creates surrogate tasks that do not require labels, enabling the model to learn representations by generating labels from the data itself and optimizing a pretext task.

Question 5

Describe the concept of an autoencoder.

Answer 5

An autoencoder is a neural network that learns to encode input data into a lower-dimensional representation and then decodes it back to the original data, minimizing reconstruction error.

Question 6

What is a variational autoencoder (VAE)?

Answer 6

A VAE is a type of autoencoder that uses variational inference to encode input data into a probabilistic latent space and decode it, allowing for both data reconstruction and generation.

Question 7

Explain the role of the Evidence Lower Bound (ELBO) in VAEs.

Answer 7

ELBO is a variational bound used to approximate the likelihood of the data, allowing the optimization of the VAE by maximizing this bound instead of the intractable true likelihood.

Question 8

What is a Generative Adversarial Network (GAN)?

Answer 8

A GAN consists of two neural networks, a generator and a discriminator, that compete in a minimax game, with the generator creating realistic data samples to fool the discriminator, which tries to distinguish between real and fake samples.

Question 9

Describe the concept of contrastive learning.

Answer 9

Contrastive learning involves training models to distinguish between similar and dissimilar pairs of data, encouraging the model to learn representations that bring similar samples closer and push dissimilar samples apart.

Question 10

What is the purpose of a latent variable model?

Answer 10

Latent variable models assume that data lies in a lower-dimensional subspace and build a mapping between the high-dimensional data space and this latent space, facilitating tasks like data generation and reconstruction.

Question 11

Explain the main idea behind flows in unsupervised learning.

Answer 11

Flows use invertible transformations to map between data space and latent space, allowing for exact likelihood computation and sampling without the need for approximation.

Question 12

What is the challenge with high-dimensional data in unsupervised learning?

Answer 12

High-dimensional data requires more samples to effectively learn and model the underlying distribution, making it computationally challenging and necessitating advanced techniques to handle the dimensionality.

Question 13

Write the formula for the reconstruction loss in an autoencoder.

Answer 13

Question 14

Provide the formula for the variational lower bound (ELBO) in VAEs.

Answer 14

Question 15

What is the formula for the contrastive loss?

Answer 15

Question 16

How does a VAE differ from a traditional autoencoder?

Answer 16

A VAE encodes input data into a probabilistic latent space using variational inference, allowing for both data reconstruction and generation, while a traditional autoencoder uses a deterministic latent space.

Question 17

Explain the role of the generator in a GAN.

Answer 17

The generator in a GAN creates realistic data samples from random noise, aiming to fool the discriminator into classifying the generated samples as real.

Question 18

What are the advantages of using self-supervised learning?

Answer 18

Self-supervised learning allows models to learn useful representations from unlabeled data, reducing the dependency on large labeled datasets and enabling transfer learning to other tasks.

Question 19

Describe the concept of a surrogate loss in self-supervised learning.

Answer 19

A surrogate loss is a loss function created from the data itself, allowing the model to learn meaningful representations by solving a pretext task without requiring labeled data.

Question 20

What is the significance of the latent space in VAEs and GANs?

Answer 20

The latent space represents a lower-dimensional subspace where the data distribution is modeled, facilitating tasks like data generation, interpolation, and reconstruction.

Question 21

How do flows differ from VAEs in unsupervised learning?

Answer 21

Flows use invertible transformations to map between data space and latent space, allowing for exact likelihood computation and sampling without approximation, while VAEs rely on variational inference.

Question 22

What is the importance of contrastive learning in unsupervised representation learning?

Answer 22

Contrastive learning helps models learn robust representations by distinguishing between similar and dissimilar pairs, improving the quality of learned features for downstream tasks.

Question 23

How does the jigsaw puzzle pretext task work in self-supervised learning?

Answer 23

The jigsaw puzzle pretext task involves shuffling patches of an image and training a model to predict the correct arrangement, encouraging the model to learn spatial relationships and features.

Question 24

Explain the concept of predicting image rotations in self-supervised learning.

Answer 24

Predicting image rotations involves training a model to identify the rotation angle of an image, helping the model learn orientation-invariant features and improving representation learning.

Question 25

What is the main challenge with kernel density estimation (KDE) in high-dimensional data?

Answer 25

KDE requires a large number of samples to effectively fill the high-dimensional space, making it computationally infeasible for high-dimensional data like images.

Question 26

Describe the main idea behind hierarchical VAEs.

Answer 26

Hierarchical VAEs use multiple layers in the latent space to capture complex data distributions, improving the quality and realism of generated samples compared to standard VAEs.

Question 27

How does the contrastive loss function help in unsupervised learning?

Answer 27

The contrastive loss function encourages the model to learn representations that bring similar samples closer and push dissimilar samples apart, improving the quality of learned features.