Handout #9 - Generative Models

Question 1

Explain what a discriminative model is:

Answer 1

1. Look at the likelihood P(Y|X) -> use pre-existing data
2. Supervised learning models (training data X is associated with given labels y)
3. ‘Given a dataset of labelled images, detect whether the picture is of a face or not.’

Question 2

Explain what a generative model is:

Answer 2

1. Look at the likelihood P(X|Y) -> make new data
2. Unsupervised learning
3. ‘Given a dataset of faces, and you would like to generate new realistic looking faces.’

Question 3

Why is training a GAN network difficult?

Answer 3

If the generator is perfect (the synthetic and data samples are the same), then the discriminator will be a bit random and at the next step the generator will not be put in check anymore by the discriminator.

Question 4

How can the GAN be trained

Answer 4

1. Freeze the generator and train the discriminator.
2. Freeze the discriminator and train the generator.
3. Repeat steps 1. and 2.

Question 5

Explain what an autoencoder does

Answer 5

Autoencoders simply try to reconstruct the input as faithfully as possible.

Question 6

Explain what the multi-task training strategy is.

Answer 6

1. Encoder is key part of the autoencoder architecture.
2. Train it through supervised learning.
3. Strategy; combine multiple approaches in a multi-task training strategy.
4. Same encoder can be shared across a multitude of classification tasks that are related to the application at hand.