Generative Adversarial Network

Question 1

What are the two components of an Autoencoder and their purpose?

Answer 1

The encoder and the decoder. The encoder takes input and downsizes it. The decoder takes the downsized input and tries to reconstruct it as close the the original input as possible

Question 2

What is the difference between a variational autoencoder and a basic autoencoder

Answer 2

Instead of only learning to reconstruct images, a variational autoencoder learns the data distribution of the training data. The latent space is, therefore, distributions and the encoder samples from this distribution to reconstruct.

Question 3

What are the two losses in a variational autoencoder?

Answer 3

The L2 loss (How similar the output image is from the input image), and the KL divergence (The distance of the latent space distributions of z from N(0, 1))

Question 4

What can you use a variational autoencoder for?

Answer 4

Compression, generation,

Question 5

What are some of the problems of a variational autoencoder?

Answer 5

You cannot control which features of the training data is represented by the latent space z, neither the position.

Question 6

Explain the intuition behind a GAN

Answer 6

Two networks, generator, and discriminator. The generator generates eg. images and its purpose is to “fool” the discriminator. The discriminator then tries to distinguish between the generated “false” images from the generator and some real images provided.

Question 7

What is the problem of using KL loss function in GANs? What is the solution? Hint: WGAN - explain this

Answer 7

The KL loss function measures the “closeness” of two distributions, i.e how similar they are, this is directly related to how they overlap. However, if the two distributions different entirely, i.e no overlap, then we have close to zero loss, which ruins all further training.

The solution is using WGAN, i.e Wasserstein GANs which is a new loss function for our GAN. This loss function is also called “earth movers distance” since it tries to move a pile of dirt a mean distance in order to look as similar as possible to another pile of dirt. The pile of dirts are naturally our probability distributions.

Question 8

Briefly explain the CycleGAN

Answer 8

Create two discriminators and two generators. One generator is a transform from A to B whilst the other generator is B to A (example: horse to zebra, zebra to horse). Then one discriminator for each domain. Also add loss to make the original horse before horse to zebra and zebra to estimated horse as close as possible