Generative Models Flashcards
Notes on Generative Models that might help with the exam.
What are the two methods for implementing Generative Models?
Variational Autoencoders (VAEs)
Generative Adversarial Networks (GANs)
What is the general structure of a VAE?
Encoder-decoder structure, constructed by neural networks. The input goes through an Encoder, to become a latent space, then that is passed through a Decoder to produce the output.
Why is regularisation important in the context of VAEs?
Without it, the encoder-decoder model can be overfitted to the input data and reconstruct very accurate examples of the training data, but it won’t be able to generate new examples.
How is regularisation implemented in VAEs?
The encoder learns to map the input data into a Gaussian distribution, which contains the mean and the covariance matrix
What is the structure of the loss function used in VAEs?
Reconstruction term - Makes the generated output as similar as to the input
Regularisation term - Makes the learned latent features from the encoder as close as possible to a Gaussian distribution.
How does a VAE generate new objects/images?
They are generated by sampling the mean and covariance of the Gaussian Distribution, and then inputting that into a decoder.
What is the effect of larger regularisation in VAEs?
Larger regularisation makes the reconstruction error larger, hence reconstructed data is less realistic/more blurry.
What are the advantages of VAEs?
Learned latent space is well constrained and easy to sample from
Easier to train compared to other generative models - Smaller training set
Broad application scenarios e.g. anomaly detection, data synthesis
What is the disadvantage when using VAEs?
The generated images tend to be on the blurry side
What are some primary applications for GANs?
Super-resolution
Image in-painting
Image synthesis
Text-to-image
What is the general structure of GANs?
Generator - Learns to generate plausible instances that look realistic. These become negative training examples for the discriminator
Discriminator - Learns to distinguish the generator’s fake data from real data, which penalises the generator for producing unreal examples
Both parts are neural networks, which are trained iteratively.
What is the Discriminator in regards to GANs?
The discriminator is a type of classifier, which could use any network architecture appropriate to the type of data it’s classifying
What happens during the training stage for the Discriminator in regards to GANs?
The Discriminator classifies both real and fake data
The Discriminator loss penalises the Discriminator for misclassifying a real and a fake instance
The Discriminator updates its weights through backpropagation from the Discriminator loss through the Discriminator network
What is the Generator, in regards to GANs?
The Generator part learns to create fake data by incorporating feedback from the Discriminator. It learns to make the Discriminator classify its outputs as real.
What happens during the training stage of the Discriminator?
During training:
- Input random noise
- The input goes through the Generator’s network to generate a data instance
- The Discriminator classifies the generated data with an output
- Calculate the loss from the Discriminator classification, which penalises the generator for failing to fool the Discriminator based on the output
- Backpropagate through both the Discriminator and Generator to obtain gradients
Use gradients to change only the Generator’s weights, but don’t touch the Discriminator
