lecture 11 Flashcards
What is the goal of generative modeling?
To train probability models that allow us to sample realistic data points.
Why are generative models important?
They can produce high-dimensional objects like images and text that resemble real data.
What is a common application of generative models?
Generating realistic human faces using neural networks.
What is a neural network in the context of generative models?
A function that maps an input (often random noise) to an output resembling real data.
What does a standard multivariate normal distribution represent in generative models?
A simple distribution from which we sample inputs for a generator network.
What is the first approach to turning a neural network into a probability distribution?
Interpreting the network’s output as the parameters of a probability distribution.
How does a neural network become probabilistic?
By treating its output as defining a probability distribution over the output space.
What distribution is typically used for binary classification?
The Bernoulli distribution.
What distribution is used in multiclass classification with a softmax output?
The multinomial distribution.
What is a key insight from linear regression?
Interpreting the output as the mean of a normal distribution leads to least squares loss.
What does the likelihood function measure in generative models?
The probability of observed data given the model’s parameters.
What is the maximum likelihood principle?
Optimizing model parameters to maximize the probability of observed data.
What is a loss function derived from likelihood maximization?
Negative log-likelihood loss.
What is binary cross-entropy loss used for?
Training models with a Bernoulli output distribution.
What is categorical cross-entropy loss used for?
Training models with a softmax output in multiclass classification.
What is mean squared error (MSE) loss used for?
Training models with normal output distributions.
What is the difference between MSE and mean absolute error (MAE)?
MSE penalizes large errors more due to squaring the differences, while MAE treats all errors equally.
What is the benefit of using a probabilistic approach in neural networks?
It provides a measure of uncertainty along with predictions.
Why are neural networks poor at estimating uncertainty?
They often produce overconfident predictions due to high complexity.
What is an alternative to single-value predictions in generative models?
Producing both a mean and variance to model a full probability distribution.
What is a multivariate normal distribution used for?
Generating high-dimensional outputs, such as images.
What is a key challenge in generative modeling?
Ensuring that generated samples are diverse and realistic.
What is an example of a deep generative model?
Generative Adversarial Networks (GANs).
What is the role of a generator network in a GAN?
To produce realistic samples that resemble real data.
What is the role of a discriminator network in a GAN?
To distinguish between real and generated samples.
What is the objective of a GAN?
To train the generator to produce samples that the discriminator cannot distinguish from real data.
What is the adversarial training process?
A competition between the generator and discriminator where each improves iteratively.
What is the main advantage of generative models?
They can learn complex data distributions without explicitly defining them.
