Topic 4 Fitting probability Distributions Flashcards
What is statistical inference?
Inferring the nature of a population distribution based on a sample.
Fitting a probability model to data is known as _______.
statistical inference
Match summary stats ↔ ______
Maximise likelihood ↔ ______
Method of Moments (MoM)
Maximum Likelihood Method (MLM)
What is the main principle of the Method of Moments?
Match population moments (mean, variance, etc.) to sample moments.
METHOD OF MOMENTS (MoM)
E(X) = __
Var(X) = __
x̄
s²ₓ
Minimise weighted difference:
Min_θ ∑ₖ wₖ (Mk − mk)²
Where:
Mk = __
mk = __
- Population moment
- Sample moment
Estimated parameters are denoted with a ___.
hat (e.g. ω̂, ε̂)
If x̄ and s²ₓ are sample mean and variance:
p̂ = ____
n̂ = ____
- 1 − s²ₓ / x̄
- x̄² / (x̄ − s²ₓ)
Geometric MoM Estimator
p̂ = ____
x̄ / s²ₓ
Gamma MoM Estimators
ε̂ = _____
ω̂ = _____
- s²ₓ / x̄
- x̄² / s²ₓ
What does MLM aim to do?
Find the parameter values that maximise the probability of observing the sample
Formula
Likelihood function:
L(θ) = ∏ fₓ(xᵢ | θ)
Condition for Maximum
First derivative =
Second derivative <
0
Why use log-likelihood?
- Easier to differentiate
- Converts product to sum
Log-Likelihood
Formula
ln(L) = ∑ ln(fₓ(xᵢ | θ))
List 3 graphical methods to assess fit:
- Empirical histogram vs PMF/PDF
- Empirical vs theoretical CDF
- QQ plot
In QQ plots, the theoretical quantile is plotted against the ______ quantile.
Empirical
QQ Plot Formula
p = (j − 0.5)/n → used to find ______ quantiles
theoretical
x̄ → E(X) as n → ∞ is known as the ______
Law of Large Numbers
What makes an estimator efficient?
It has the lowest possible variance (Cramér-Rao lower bound)
Use s²ₓ instead of σ²ₓ and ĝ₁(x) instead of g₁(x) to ensure _______ sample moments.
Unbiased
What are the parameter estimators obtained by MoM and MLM?
- Consistent
- Biased
- Inefficient
When is a an estimator consistent?
An estimator is consistent if the sample moments
converge to the population moments as the sample size
tends to infinity e.g., σ²ₓ → Var(X)
What is a biased estimator?
A biased estimator has a mean sample moment that is
not equal to the population moment
Do MoM and MLM produce efficient or inefficient estimators?
Inefficient
What does it mean by the MLM approach being asymptotically normal?
For large sample sizes the estimators are normally-distributed random variables
