Sums of independent random variables and limit theorems

Question 1

Convolution of pₓ and py

Answer 1

The distribution of X+Y with X and Y are real valued, independent discrete random variables

P(X+Y = Z) = Σₓ P(X=x, Y=Z-X)
P(X+Y = Z) = Σₓ P(X=x)* P(Y=Z-X)

pₓ+y(z) = Σₓ pₓ(x) py(z-x)

fₓ+y(z) = -∞∫∞ fₓ(x)*fy(z-x) dx

Question 2

Gamma random variable

Answer 2

X~Γ(α, λ) α, λ>0

     | (λᶛ /Γ(α))*x^(α-1)*e^(λx)   x>0 fₓ(x)  |
     |0                       otherwise

Where Γ(α) is the so-called Euler Γ-function

Γ(α)= ₀∫∞ x^(α-1)*e^(λx) dx

In particular if X₁, X₂, …, Xₙ ~ Exp(λ) is independent then

Xᵢ ~Γ(1, λ) ,
X₁, X₂, …, Xₙ ~Γ(n, λ)

Question 3

Law of large number

Answer 3

Consider X₁, X₂, …, Xₙ random variables, real valued independent and with the same distribution (iid)

E(Xᵢ) = E[(X₁, X₂, …, Xₙ)/n] = µ

Var(Xᵢ) = ͳ² * (1/n)

So the distribution of Xₙ gets more and more concentrated around of µ as
n –> + ∞

Question 4

Law of large number
▪ Theorem

Answer 4

For every ε>0

lim P(|Xₙ - µ|)⩾ε) = 0

Roughly speaking

Xₙ = µ for n large

Question 5

Law of large number
▪ Error

Answer 5

E(X₁ + X₂ + … + Xₙ) = nµ
Var(X₁ + X₂ + … + Xₙ) = nͳ²

Define Zₙ = (X₁ + X₂ + … + Xₙ) / (ͳ * √n)

E(Zₙ) = 0
Var(Zₙ) = 1

Remark
▪ Suppose
X~N(µ, ͳ²)
Then
Zₙ~N(0, 1)

▪ Zₙ = (Xₙ - µ) / (ͳ * √n)
So
Xₙ = µ + Zₙ * (ͳ /√n)

Where Zₙ * (ͳ /√n) is the error

Question 6

Central limit theorem

Answer 6

As n –> ∞ the distribution of Zₙ approaches the distribution N(0, 1)

lim P(Zₙ ⩽ x) = Ф(x) ∀x

Ф(x) –> Distrib. function of a N(0, 1)

Question 7

Central limit theorem
▪ Normal approximation

Answer 7

P(a⩽ (X₁ + X₂ + … + Xₙ) ⩽ b =
= P(a - nµ / (ͳ * √n) ⩽ (X₁ + X₂ + … + Xₙ- nµ ) / (ͳ * √n) ⩽ b - nµ/ (ͳ * √n)) =
= P(a-nµ / (ͳ * √n) ⩽ Zₙ ⩽ b - nµ/ (ͳ * √n)) =
= Ф(b - nµ/ (ͳ * √n)) - Ф(a - nµ / (ͳ * √n))

If low n

P = Ф(b + 1/2 - nµ/ (ͳ * √n)) - Ф(a - 1/2 - nµ / (ͳ * √n)) –> correction of continuity

Example
X~Be(p)
need correction if
np >5
n(1-p) >5