Monte Carlo - Independent Flashcards
What is the inverse probability integral transform method?
transform uniform RV using inverse of the cdf of Y, g(U) = Fy^(-1)(y)
i.e, set u = Fy^(-1)(y), solve for y
Explain rejection sampling
∫p(θ) dθ is estimated by the fraction of darts in the shaded region * area of the rectangle
Give the method for general rejection sampling
- p(θ) target, g(θ) envelope where the support of g includes the support of p
- find m ≥ p(θ) / g(θ) for all θ
- Generate θ* from g(θ)
- Generate u from uniform(0, m*g(θ))
- If u ≤ p(θ) keep θ, else go back to 1
What is the acceptance rate in general rejection sampling?
1/m
Evaluate rejection sampling
+ very general so can be used for any distribution as long as the target can be calculated
+ easy to implement
+ don’t need a constant of proportionality
+ if can find an envelope where m is small, can be very efficient
- if for envelope m is large, very inefficient
- can be hard to find a good envelope, even for just three dimensions
Key point about rejection sampling
If g(θ) roughly proportional to p(θ), has similar shape, very efficient
Describe importance sampling
independent samples from the wrong distribution but values are adjusted to be from the right distribution NO REJECTION
Give the method for importance sampling
Ep( h(θ|y)) = Eg(h(θ) * w(θ))
= 1/N * (sum to N) h(θi) * w(θi)
where w(θ) = p(θ|y) / g(θ)
Describe sampling importance resampling
Generated θ and importance ratios wi can be used to produce an approximate sample from target distribution p(θ|y)
Give the method for generating a sample of size n with sampling importance resampling
- Choose N>n, generate N samples of θi from importance sampler g(θ)
- Calculate importance ratio and scale them by the sum of the weights
wi* = wi / sum wj - Randomly sample n θi with replacement with probabilities wi*
