Generative models VAEs and GANs

Question 1

What are generative models? What is implicit denstiy estimation and explicit density estimation?

Answer 1

Generative models: learn a model so that pmodel(x) is as close as pdata(x)
Explicit density estimation: Explicity define and solve a model pmodel(x)
Markov chains or VAE
İmplicit denstiy estimation: learn a moel which can sample from pmodel without explicity defining it.

Question 2

Differentiable Generator Networks

Answer 2

parameterized functions for generating samples
A differentiable function g(z;model) transforming the samples of latent variables z to
direct samples x
or indirect distributions over samples
Typically represrented by a neual netwokr.
Examples of direct samples: drawing samples x from a normal distribution with mean and covariance
Examples of indirect samples: use g(.) with sigmoid outputs to provide the mean parameters of bernoulli distributions

Question 3

VAE s

Answer 3

assume that the training data T{xi} is generated from the latent variables z
z ->(decoder networks) x
Goal: estimate the true parameters of this generative model
Solutioın: pick a simple distribution for prior p(z) e.g. gaussian
use a generator network for the posterior p(x|z)
need a maximum liklihood of p(x)

enoder -> mean and variance -> sample from distributions -> decoder

Data generation:
sample z from z’
decoder network mean and variance
sample x|z from x|z ‘

Question 4

Gans

Answer 4

Unlike Vae we dont need to define an explicit model, instead it learns from the training data to draw samples.
input: z(random noise) -> geenrator network -> real and fake data -> discriminative network -> real or fake
GRADIENT ASCENT on discriminator
GRADIENT DESCEND on generator

Question 5

Normalizing Flows

Answer 5

Define expressive probability distributions onl y requiring the spesification of a base distribution
a series of invertible transformations