week 3 formatted Flashcards
What is the probability density function (pdf) of a Normal distribution N(μ, σ²)?
f(x|μ, σ²) = (1 / √(2πσ²)) * exp[-(x - μ)² / (2σ²)]
Why is the Normal distribution particularly important in statistics, according to the notes?
Mainly due to the Central Limit Theorem (CLT) and its analytic tractability, especially in Bayesian contexts.
In frequentist inference for a N(μ, σ²) model based on an i.i.d. sample X₁, ..., Xn, what are the distributions of the sample mean X̄ and the statistic (n-1)S²/σ²?
X̄ ~ N(μ, σ²/n) and (n-1)S²/σ² ~ χ²(n-1), where S² is the sample variance.
What is the relationship between the sample mean X̄ and the sample variance S² for an i.i.d. sample from a Normal distribution?
X̄ and S² are independent.
What statistic involving X̄, μ, S, and n follows a t-distribution, and what are its degrees of freedom?
(X̄ - μ) / (S / √n) ~ t(n-1)
In the Bayesian model for N(μ, σ²) with σ² known (equal to σ₀²), what is the conjugate prior for μ?
A Normal distribution, π₀(μ|σ₀²) = Normal(η, σ₀²/λ).
For the Normal-Normal model (N(μ, σ₀²) likelihood, N(η, σ₀²/λ) prior), what is the posterior distribution for μ, π(μ|x, σ₀²)?
Normal(η_n, σ₀²/λ_n), where η_n = (n*X̄ + λη)/(n+λ) and λ_n = n + λ.
Interpret the posterior mean ηn in the Normal-Normal model.
It is a weighted average of the sample mean X̄ (weighted by n) and the prior mean η (weighted by λ).
What happens to the posterior distribution for μ in the Normal-Normal model as the prior precision λ approaches 0?
The posterior approaches Normal(X̄, σ₀²/n), which corresponds to the frequentist sampling distribution centered at the MLE X̄.
In the Bayesian model for N(μ, σ²) with both parameters unknown, how is the prior typically factorized?
π₀(μ, σ²) = π₀(μ|σ²) * π₀(σ²)
What are the standard conjugate prior choices for π₀(μ|σ²) and π₀(σ²) when the likelihood is N(μ, σ²)?
π₀(μ|σ²) = Normal(η, σ²/λ) and π₀(σ²) = InverseGamma(α/2, β/2).
When both μ and σ² are unknown, what is the marginal posterior distribution for μ, π(μ|x), obtained by integrating out σ²?
A non-standardized Student’s t-distribution: μ|x ~ t(μ_post, S²_post/(λ+n), n+α).
Provide the formula for the location parameter μ_post of the marginal posterior t-distribution for μ.
μ_post = (n*X̄ + λη) / (n+λ)
What is the marginal posterior distribution for σ², π(σ²|x), obtained by integrating out μ?
InverseGamma(αn, βn), where αn = (n+α)/2.
Provide the formula for the shape parameter αn of the marginal posterior Inverse Gamma distribution for σ².
αn = (n+α)/2
Provide the formula for the scale parameter βn of the marginal posterior Inverse Gamma distribution for σ².
βn = (1/2) * [β + Σ(xi - X̄)² + (nλ/(n+λ))(X̄ - η)²]
Can posterior samples for μ and σ² be drawn independently from their respective marginal posteriors π(μ|x) and π(σ²|x)?
Yes.
Alternatively to using marginal posteriors, how can we obtain posterior samples using conditional distributions (e.g., for Monte Carlo methods)?
Iteratively sample σ² from π(σ²|x) and then sample μ from π(μ|σ², x), OR sample μ from π(μ|x) and then sample σ² from π(σ²|μ, x).
What is the conditional posterior distribution of μ given σ² and data x, π(μ|σ², x)?
Normal(μ_post, σ²/(n+λ)), where μ_post = (n*X̄ + λη) / (n+λ).
What is the conditional posterior distribution of σ² given μ and data x, π(σ²|μ, x)?
InverseGamma(αn', βn'), where αn' = (n+α+1)/2 and βn' = (1/2) * [β + Σ(xi - μ)² + λ(μ - η)²]. (Note: Formula in pdf uses μ_post but this form using μ` directly is equivalent).
How is the marginal posterior for μ, π(μ|x), calculated from the joint posterior π(μ, σ²|x)?
π(μ|x) = ∫<sub>[0, ∞]</sub> π(μ, σ²|x) dσ²
How is the marginal posterior for σ², π(σ²|x), calculated from the joint posterior π(μ, σ²|x)?
π(σ²|x) = ∫<sub>[-∞, ∞]</sub> π(μ, σ²|x) dμ
