1 basics Flashcards
Let f : R to R
x_0 ∈R
consider
x’(t)=f(x(t)) for t ∈ [0,T]
x(0)=x_0
Give a
A sol
A sol is a func x: [0,T] to R of class C^1 and satisfies equalities
e.g. let r>0 x’(t) = rx(t) t ∈ [0,T]
x(0)=1
unique sol
x(t) = exp(rt) t ∈ [0,T]
linear differential equation:
solve by multiplying both sides by exp(-rt) and then noting its integrating by part form to solve
exp(-rt) x(t) = 1
x(t) = exp(rt)
chain rule and ftoc
exp growth
is brownian motion differentiable?
links to avg value of stock, but the value of stock is more stochastic in nature
Brownian motion/wiener process is an example of a stochastic process
brownian motion is non differentiable at “corners” nowhere differentiable, everywhere continuous
2d brownian motion
modelled in 2 components
W_1 and W_2
A union B
{x: x∈A or x∈B}
A intersection B
{x: x∈A and x∈B}
A complement
Aᶜ:= Ω \ A ∈ F
sample space Ω
Non empty set of outcomes
subsets of the sample space Ω are EVENTS collection of all EVENTS is a SIGMA ALGEBRA
Ω COUNTABLE
X={1,2,3}
P(X)
how many elements
P(X)=
{∅,{1,2,3}, {1},{2},{3}, {1.2}, {1,3}, {2,3}}
2^3 = 8 elements incl empty set
the power set is an example of sigma algebra and is the largest example for a given X
Definition 1.1.1. A σ-algebra
Definition 1.1.1. A σ-algebra F on Ω is a collection of subsets of Ω such that
(i) Ω ∈ F (WHOLE SPACE IS AN ELEMENT)
(ii) If A ∈ F then Aᶜ:= Ω \ A ∈ F COMPLEMENTS
(iii) If Aₙ ∈ F for n ∈ N, then ∪ₙ₌₁ ∞ Aₙ ∈ F. (COUNTABLE UNIONS)
(EMPTY SET WILL BE IN)
∅ ∈ F
F σ-algebra
(EMPTY SET WILL BE IN)
Let (Ω, F) be a measurable space. Since F is a σ-algebra which is closed under complementation Ωᶜ∈ F ∅ ∈ F
σ-algebra
closed under intersections?
Show that if Aₙ ∈ F for n ∈ N, then ∩ₙ₌₁ ∞ Aₙ ∈ F
If Aₙ ∈ F for n ∈ N then Aₙᶜ ∈ F
By De-Morgan’s identity
∩ₙ Aₙ = ∩ₙ (Aₙᶜ)ᶜ = (∪ₙ Aₙᶜ )ᶜ
which is in F because σ-algebras are closed under complementation and countable union.
The pair (Ω, F)
The pair (Ω, F) is called a measurable space
measurable sets
elements of the sigma algebra
summary sigma algebra
(If i have an event A in sigma algebra I want the complement of my event also in)
(for two events A and B i want unions and intersections to be in the sigma algebra
sequences of complements and unions also contained
any sigma algebra on A is either {complement of A,A} or bigger up to P(A)
Exercise 1.1.3. Consider the set of real numbers R.
(1) Find the smallest σ-algebra on R.
{∅,R}
∅, R ∈ F by the def. of σ-algebras
Exercise 1.1.3. Consider the set of real numbers R.
(2) Find the smallest σ-algebra on R that contains the interval (0, 1).
{∅,R, (0,1), (−∞, 0] ∪ [1, ∞)}
∅, R ∈ F by the def. of σ-algebras
since σ-algebras are closed under complementation
Def σ(A)
1.1.4
Let Ω be a non empty set and let A be a collection of subsets of Ω. We denote
by σ(A) the intersection of all σ-algebras on Ω that contain A, that is
σ(A) := ∩_{B∈Sₐ}B,
where Sₐ = {B : B is a σ-algebra on Ω and
A ⊂ B}.
This is a sigma algebra
The σ-algebra generated by A
σ(A)
intersection of all s.t they contain A
smallest σ-algebra that contains A.
A is a subset of the power set on Ω
Exercise 1.1.5. Show that S_A is non empty.
S_A:= {B : B is a σ-algebra and A ⊂ B}
Let Ω be nonempty set and let A be a collection of subsets of Ω.
Let S_A:= {B : B is a σ-algebra and A ⊂ B}
Then P(Ω) ∈ S_A, because P(Ω) is a σ-algebra, and A ⊂ P(Ω). Hence S_A is non empty.
ex 1.1.5 Show that intersection of σ-algebras is a σ-algebra. Consequently, σ(A) is indeed a σ-algebra.
Let H be a family of σ-algebras.
Then
1) by first axiom Ω ∈ F for all F ∈ H. Therefore
Ω ∈ ∩_{F∈H} F
2) Let A ∈ ∩_{F∈H} F
Then A ∈ F ∀F ∈ H.
So since σ-algebras are closed under complementation:
A^c ∈ F ∀F ∈ H.
Hence
A^c ∈ ∩_{F∈H}F.
So ∩_{F∈H} F is closed under complementation.
3) Let A_1,A_2,… ∈ ∩_{F∈H} F
Then
A_1,A_2,… ∈ F ∀F ∈ H.
So since σ-algebras are closed under countable union:
∪ₙ Aₙ ∈ F ∀F ∈ H.
Hence
∪ₙ Aₙ ∈ ∩{F∈H} F.
So ∩{F∈H} F is closed under countable union.
By 1,2,3: ∩_{F∈H} F is a σ-algebra.
e.g Ω ={1,2,3,4,5}
A={{1} {1,2}, {4,5}}
σ(A)
is there a sigma algebra F s.t A contained in F and if G is another sigma algebra s.t contains A then F contained in G
σ-algebra generated by A
{∅,{1,2,3,4,5},
{1} {1,2}, {4,5},
{2,3,4,5}, {3,4,5}, {1,2,3},
{1,2,4,5}}
{3} {1,4,5}
e.g Ω =R
A={[0,1]}
is there a sigma algebra F s.t A contained in F and if G is another sigma algebra s.t contains A then F contained in G
σ-algebra generated by A
{∅,R, (0,1), (−∞, 0] ∪ [1, ∞)}
Definition 1.1.6. The Borel σ-algebra
The Borel σ-algebra on Rᵈ, denoted by B(Rᵈ) is the σ-algebra generated by the
collection of open sets of Rᵈ
“Consider a sigma algebra generated by A”
A ={u in R s.t U is open}
countable unions of open intervals
The borel sigma algebra is the smallest sigma algebra containing open sets
Exercise 1.1.7. Show that B(R) contains all sets of the form (a, b), (a, b],[a, b), [a, b], (−∞, a],
(−∞, a), (a, +∞), [a, +∞), for a, b ∈ R. Moreover, show that the σ-algebra B(R) is generated
by any of these classes
A_1: collection of all open intervals on R
A_2 collection of closed intervals
A_3: collection of (−∞, a]
A_4: collection of (−∞, a)
A_5: collection of (a,+∞)
A_6: collection of [a,+∞)
A_7: collection of closed subsets of R
Then we can show the sigma algebra generated by open sets is the same as the one generated by all of these.
We show σ(A_1)= σ(A_2)……
Method from lectures
Exercise 1.1.7. Show that B(R) contains all sets of the form (a, b), (a, b],[a, b), [a, b], (−∞, a],
(−∞, a), (a, +∞), [a, +∞), for a, b ∈ R. Moreover, show that the σ-algebra B(R) is generated
by any of these classes
To show ** σ(A_1) = σ(A)**
** σ(A_1) ⊆ σ(A)**
Take a ∈ A_1 (a will be an open interval) and is therefore an open set) thus a ∈ A. Thus A_1⊆A. By defn, σ(A) is the smallest σ-algebra containing A. A ∈ σ(A). A_1 ⊆A ⊆σ(A)
Claim σ(A_1)⊆ σ(A), by defn A_1 is contained in A thus any sigma algebra containing A_1 will have this σ(A_1) contained. As σ(A) contains A_1 we have this true.
** σ(A) ⊆ σ(A_1)**
Take an open set A ∈ A_1 This means A is a countable union of open intervals A= ∪ₙ,∞ (aₙ,bₙ)
(aₙ,bₙ) is an open interval, belongs to A_1 and thus σ(A_1).
Thus we have a collection of elements that belong to a sigma algebra, since σ(A_1) is a sigma algebra the union is an element of a sigma algebra. ∪ₙ,∞ (aₙ,bₙ)∈ σ(A_1) . Thus A ⊆ σ(A_1).
Now we have a class contained in the sigma algebra, by defn σ(A_1) is the smallest such sigma algebra, thus σ(A⊆ σ(A_1)
Method in notes
Exercise 1.1.7. Show that B(R) contains all sets of the form (a, b), (a, b],[a, b), [a, b], (−∞, a],
(−∞, a), (a, +∞), [a, +∞), for a, b ∈ R. Moreover, show that the σ-algebra B(R) is generated
by any of these classes
(i) Recall that B(R) is defined to be the σ-algebra generated by the collection of all open sets of
R. Hence B(R) contains all open sets of R. Thus (a, b) ∈ B(R).
(ii) (a, b] = ∩_n=1,∞ (a, b +1/n) ∈ B(R), since open sets are Borel, and σ-algebras are closed under
countable intersection.
(iii) [a, b) =∩_n=1,∞ (a − (1/n), b) ∈ B(R) by the same reasoning.
(iv) [a, b] = ∩_{n=1} ∞ (a −(1/n), b + (1/n) ) ∈ B(R) by the same reasoning.
(v) (−∞, a] = ∪_{n=1}∞ [−n, a] ∈ B(R) since (by (iv)) closed intervals are Borel and σ-algebras
are closed under countable union.
(vi) (−∞, a) = ∪_{n=1} ∞ (−n, a) ∈ B(R) since open sets are Borel and σ-algebras are closed under
countable union.
(vii) (a, ∞) =∪_{n=1} ∞ (a, n) ∈ B(R) by the same reasoning as in (vi).
(viii) [a, ∞) = ∪_{n=1} ∞ [a, n] ∈ B(R) by the same reasoning as in (v).
Hence B(R) contains all sets of the form(a, b), (a, b], [a, b), [a, b], (−∞, a], (−∞, a), (a, ∞), [a, ∞).
We will now show that any of these classes generates B(R)
Method in notes
Exercise 1.1.7. Show that B(R) contains all sets of the form (a, b), (a, b],[a, b), [a, b], (−∞, a],
(−∞, a), (a, +∞), [a, +∞), for a, b ∈ R. Moreover, show that the σ-algebra B(R) is generated
by any of these classes
continued
(I) Since B(R) contains all sets of the form (a, b), and σ({(a, b) : a < b}) is the smallest
σ-algebra containing all sets of the form (a, b), we have σ({(a, b) : a < b}) ⊂ B(R).
Since every open set in R is a countable union of (disjoint) open intervals, and σ-algebras
are closed under countable union, it follows that σ({(a, b) : a < b}) must contain all open
sets in R. Since B(R) is the smallest σ-algebra containing all open sets of R, we have
B(R) ⊂ σ({(a, b) : a < b})
Therefore we conclude that
σ({(a, b) : a < b}) = B(R)
(II) Similarly, as B(R) contains all sets of the form (a, b], and σ({(a, b] : a < b}) is the smallest
σ-algebra containing all sets of the form (a, b], we have σ({(a, b] : a < b}) ⊂ B(R)
Since (a, b) = ∪_n=1 ∞ (a, b − {b − a}/{2n} ]
,
every open interval is a countable union of intervals of the form (a, b]. So since σ-algebras
are closed under countable union, all open intervals must be contained in σ({(a, b] : a < b}).
So since σ({(a, b) : a < b}) = B(R) is the smallest σ-algebra containing all open intervals,
we have B(R) ⊂ σ({(a, b] : a < b}).
Hence we conclude that
B(R) = σ({(a, b] : a < b})
(III) Similarly:
σ({[a, b) : a < b}) ⊂ B(R)
and since
(a, b) = ∪_{n=1} ∞ [a + {b − a}/{2n} , b)
we also have that
B(R) ⊂ σ({[a, b) : a < b}).
Therefore
σ({[a, b) : a < b}) = B(R).
(IV) Similarly: σ({[a, b] : a < b}) ⊂ B(R)
and since (a, b) = ∪_{n=1} ∞ [a + (b − a)/(3n), b − {b − a}/3n]
we also have
B(R) ⊂ σ({[a, b] : a < b})
Therefore
σ({[a, b] : a < b}) = B(R)
(V) We have
σ({(−∞, a] : a ∈ R}) ⊂ B(R).
Furthermore (a, b] = (−∞, b] \ (−∞, a],
so since σ-algebras are closed under taking differences of sets, σ({(−∞, a] : a ∈ R}) must
contain all sets of the form (a, b]. So since σ({(a, b] : a < b}) = B(R) is the smallest
σ-algebra containing all sets of the form (a, b], it follows that
B(R) ⊂ σ({(−∞, a] : a ∈ R}).
Hence we conclude that σ({(−∞, a] : a ∈ R}) = B(R)
(VI) Similarly,
σ({(−∞, a) : a ∈ R}) ⊂ B(R),
furthermore
[a, b) = (−∞, b) \ (∞, b)
and thus
σ({[a, b) : a < b}) ⊂ σ({(−∞, a) : a ∈ R}),
i.e.
B(R) ⊂ σ({(−∞, a) : a ∈ R}).
So we conclude that
B(R) = σ({(−∞, a) : a ∈ R})
(VII) Similarly,
σ({(a, ∞) : a ∈ R}) ⊂ B(R),
furthermore
(a, b] = (a, ∞) \ (b, ∞)
and thus
σ({(a, b] : a < b}) ⊂ σ({(a, ∞) : a ∈ R})
i.e.
B(R) ⊂ σ({(a, ∞) : a ∈ R}).
So we conclude that
σ({(a, ∞) : a ∈ R}) = B(R).
(VIII) Similarly,
σ({[a, ∞) : a ∈ R}) ⊂ B(R),
furthermore
[a, b) = [a, ∞) \ [b, ∞)
and thus
σ({[a, b) : a < b}) ⊂ σ({[a, ∞) : a ∈ R})
i.e.
B(R) ⊂ σ({[a, ∞) : a ∈ R}).
So we conclude that
σ({[a, ∞) : a ∈ R}) = B(R).
This finishes the proof.
Note: In our proof we used the fact that σ-algebras are closed under taking differences of sets. This can be seen as follows: Let A, B be measurable sets. Then A \ B = A ∩ B^c
is measurable, since σ-algebras are closed under complementation and (countable and thus also finite) intersections
σ-algebras are closed under taking differences of sets?
Note: In our proof we used the fact that σ-algebras are closed under taking differences of sets. This can be seen as follows: Let A, B be measurable sets. Then A \ B = A ∩ B^c
is measurable, since σ-algebras are closed under complementation and (countable and thus also finite) intersections
Borel sigma algebra considered
On R^d we will always consider the Borel σ-algebra B(R^d).
we choose whichever generates the set most convenient for us
Def 1.1.8
F/F’ measurable
Let (Ω, F) and (Ω’, F’) be two measurable spaces. A function f : Ω → Ω’ is called F/F’-measurable if f⁻¹(B) ∈ F for all B ∈ F’
(If I take any set in F’,B, the inverse image of this set under f is the set)
f⁻¹(B) ={w∈Ω| f(w) ∈B}
e.g. suppose we have function f mapping: from c to Ω’
ω₁→1
ω₂→3
ω₃→2
ω₄→1
f⁻¹({1,2}) =
when is it measurable F/F’ measurable
f⁻¹({1,2}) ={ω₁,ω₃,ω₄}
For the function to be measurable we want this preimage to be a subset of the original Ω for every subset of Ω’
F-measurable.
Ω’ = R^d,
If Ω’ = R^d, then as already mentioned we will always consider F’ = B(R^d).
Moreover, in this
case, for a F/B(R^d)-measurable function we will simply say it is F-measurable.
e.g Let Ω={ω₁,ω₂,ω₃,ω₄}
σ-algebra F ={∅, Ω, {ω₁},{ω₂,ω₃,ω₄}}
what is this?
σ({ω₁})
ie the smallest sigma algebra generated by {ω₁}
Let Ω={ω₁,ω₂,ω₃,ω₄}
σ-algebra F ={∅, Ω, {ω₁},{ω₂,ω₃,ω₄}}
Let Ω’={1,2}
σ-algebra F’ ={∅, Ω, {1},{2}}
define function f mapping Ω to Ω’:
ω₁→1
ω₂→2
ω₃→2
ω₄→2
check if F/F’ measurable
Checking every set:
f⁻¹(∅)= ∅
f⁻¹(Ω)= f⁻¹({1,2})= {w∈Ω| f(w) ∈Ω’) } ={ω₁,ω₂,ω₃,ω₄}= Ω
f⁻¹({1})= {ω₁}
f⁻¹({2})= {ω₂,ω₃,ω₄}
These we are elements in F thus the function f
is measurable
Let Ω={ω₁,ω₂,ω₃,ω₄}
σ-algebra F ={∅, Ω, {ω₁},{ω₂,ω₃,ω₄}}
Let Ω’={1,2}
σ-algebra F’ ={∅, Ω, {1},{2}}
Define function g mapping Ω to Ω’:
ω₁→1
ω₂→1
ω₃→2
ω₄→2
check if g is F/F’ measurable
No,
clearly always
g⁻¹(∅)= ∅
g⁻¹(Ω)= f⁻¹({1,2})= {w∈Ω| f(w) ∈Ω’) } ={ω₁,ω₂,ω₃,ω₄}= Ω
g⁻¹({1})= {ω₁, ω₂} but doesn’t belong to F not an element of F
Lemma 1.1.9
for two measurable spaces
Let (Ω, F) and (Ω’, F’) be two measurable spaces. Let A’ be a collection of subsets of Ω’ and assume that F’ = σ(A’). (the sigma algebra is generated by a collection of subsets)
Then, a function f : Ω → Ω’ is F/F’ -measurable if and only
if f⁻¹(B) ∈ F for all B ∈ A’ (A’ the generating class)
——————————————
if inverse image is a subset then is measurable but in this case F’ is generated by class A so we don’t need to check the inverse for all elements of F but only of B ∈ A’, the generating class A’
In particular If target space is R and borel sigma algebra, suffices to check inverse image of one of our prev A_i is a measurable set
.
Exercise 1.1.10. Let (Ω₁, F₁), (Ω₂, F₂), and (Ω₃, F₃) be measurable spaces.
Let f : Ω₁ → Ω₂ be F₁ /F₂-measurable and
let g : Ω₂ → Ω₃ be F₂/F₃-measurable.
Show that g ◦ f : Ω₁ → Ω₃ is
F₁/F₃-measurable COMPOSITION
We know g ◦ f : Ω₁ → Ω₃ is defined on Ω₁.To show that g ◦ f : Ω₁ → Ω₃ is F₁/F₃-measurable:
Let A∈F₃. We want to show that inverse image of A belongs in F₁. (g ◦ f)⁻¹(A) = f⁻¹(g⁻¹(A)). We know g is F₂/F₃-measurable. Thus g⁻¹(A)∈F₂. Since f is F₁ /F₂-measurable f⁻¹(g⁻¹(A))∈F₁. Thus the composition is F₁/F₃-measurable:
Exercise 1.1.11. Let f : R → R be a continuous function. Show that it is B(R)-measurable (Borel measurable)
Went through (for students with some background with analysis, “don’t get too frustrated”
We need to show that f is B(R)/B(R)-measurable. Since B(R) is generated by the collection of open sets in R, recalling Lemma 1.1.9 it suffices to show that f⁻¹(A) ∈ B(R) ∀A ⊂ R such that A is open.
To this end, let A ⊂ R be open. Since f is continuous, the preimage f⁻¹(A) is also open. So since open sets are Borel, f⁻¹ (A) ∈ B(R), as required.
Consider
f:R to R
f(x)= 1
is this measurable?
(Ω, F) to (R, B(R)) the Borel sigma algebra is considered here on R
To check measurable look at inverse image f⁻¹(A) for A ∈B(R) and check it is in F. Or from the lemma check if A in the form A=(a,b).
f⁻¹(A) = {w∈Ω| f(w) ∈A } = {w∈Ω| 1 ∈A } =
{Ω if 1 ∈A
{∅ if 1 not in A
constant functions are measurable (swap 1 to constant c)
Proporition 1.1.12
f+g
Let (Ω, F) be a measurable space and let f, g : Ω → R be F-measurable functions.
Then f + g and f · g are F-measurable.
(Here we are using borel as on R)
Exercise 1.1.13. CONSIDERING SEQUENCES OF FUNCTIONS
Let (Ω, F) be a measurable space and let fₙ : Ω → R be F-measurable functions (borel measurable sequence of functs) for n ∈ N.
inf fₙ are measurable.
Similarly, since the collection of sets of the form [a, ∞) generates B(R), it suffices to show
that (infₙ fₙ)⁻¹ ([a, ∞)) ∈ F ∀a ∈ R.
Now,
(infₙ fₙ)⁻¹ ([a, ∞))
= {ω ∈ Ω : infₙ fₙ ∈ [a, ∞)}
= {ω ∈ Ω : ∀n ∈ N, fₙ(ω) ≥ a}
=∩{n∈N}{ω ∈ Ω : fn(ω) ≥ a}
=∩{n∈N} fₙ⁻¹ ([a, ∞))
which is in F since [a, ∞) is Borel, fₙ is F/B(R)-measurable, and σ-algebras are closed under countable intersection. Hence infₙ fₙ is also measurable
Exercise 1.1.13. CONSIDERING SEQUENCES OF FUNCTIONS
Let (Ω, F) be a measurable space and let fₙ : Ω → R be F-measurable functions (borel measurable sequence of functs) for n ∈ N.
(1) Show that supₙ fₙ,
Will be F measurable
𝒻~ caligraphic A
I want to check the inverse image of a borel set is a borel set
Checking : supₙ (fₙ⁻¹ (A))
From the lemma it suffices to check smaller class A∈B(R)= σ(𝒻)
We choose A ⊂𝒻 most convenient
𝒻 = {(-∞,α]| α∈R}
(supₙ fₙ)⁻¹ ((-∞,α])
= {ω ∈ Ω : supₙ fₙ(ω) ∈(-∞,α] }
= {ω ∈ Ω : supₙ fₙ(ω) ≤ α } (sup< then all are less than)
={ω ∈ Ω : supₙ fₙ(ω) ≤ α for all n∈N }
=∩ₙ {ω ∈ Ω : supₙ fₙ(ω) ≤ α }
=∩ₙ fₙ⁻¹ ((-∞,α]) ((-∞,α]⊂B(R) is a borel set; the inverse image of a borel set via a measurable function is a measurable set so fₙ⁻¹ ((-∞,α])∈F for all n in N)
Thus for a sequence of elements in the sigma algebra, the intersection will also be in
∩ₙ fₙ⁻¹ ((-∞,α]) ∈F
thus measureable
Summary from exercises for sequences of measurable functions fₙ
sup and inf
provided each fₙ is measurable
supₙ fₙ is measurable and infₙ fₙ is measurable
Thus the limit lim inf fₙ is measurable
(limₙ inf fₙ = supₖ(inf _ₙ≥ₖ fₙ )
take the infimum of fₙ for n≥k
looking at infimum of less and less things means its a increasing sequence in k
for limits of sequences of fₙ is measurable functions and the limits exists then the limit is measurable function too
limits, compisitions of measurable functions are measurable!
Exercise 1.1.13. CONSIDERING SEQUENCES OF FUNCTIONS
Let (Ω, F) be a measurable space and let fₙ : Ω → R be F-measurable functions (borel measurable sequence of functs) for n ∈ N.
(2) Show that lim supv→∞ fₙ is a measurable functionf
didnt go through in particular but mentioned it
If for each ω ∈ Ω the limit limn→∞ fₙ(ω) exists, then
f(ω) = limₙ→∞ fₙ(ω) = lim inf ₙ→∞ fₙ(ω)
= sup_{n≥0} inf_{m≥n} fₘ(ω)
which is measurable by part 1. of the exercise.
Def 1.1.1.4
PROBABILITY MEASURE
Let (Ω, F) be a measurable space. A non-negative function P : F → [0, 1] is called a probability measure if
1. P(∅) = 0
- If Aₙ ∈ F, for n ∈ N, and Aₙ ∩ Aₘ = ∅ for n ≠ m, then
P(∪ₙ₌₁∞ Aₙ) = Σₙ₌₁∞ P(Aₙ)
(disjoint union = sums of individual probabilities) - P(Ω) = 1
(Funct P is defined for arguments which are sets)
(Ω, F, P)
PROBABILITY SPACE
A measure
A non-negative function µ : F → [0, +∞] satisfying 1 and 2 of the above definition will be
called a measure
A probability space is
a particular case of a measure space
Let (Ω, F, P) be a probability space. A set A ⊂ Ω is called null, if
if there exists B ∈ F such
that P(B) = 0 and A ⊂ B
The probability space (Ω, F, P) is called complete if
The probability space (Ω, F, P) is called complete if F contains all the
null sets.
Exercise 1.1.15. Show that if A, B ∈ F and A ⊂ B, then P(A) ≤ P(B).
The probability measure gives properties for disjoint unions
so rewrite as a disjoint set union
We have that
B= A∪ (B\A) Since A and (B \ A) are disjoint, we get
P(B) = P(A) + P(B \ A). Since P(B \ A) ≥ 0,
we get P(B) ≥ P(A).
Consider a probability space modelling rolling a die
(Ω, F, P)
Ω= {1,2,3,4,5,6}
F= P(Ω)={A| A⊆Ω} power set
= {{1,2,3,4,5,6}, ∅, {1}, {2,3,4,5,6},….} 2^6 elements
P(A)= #A/6
P({2,4,6}) = 3/6=1/2
take disjoint sets union prob= sum
P( {2,3} U {5,6})= P ({2,3,5,6})= 4/6 or sum of them
Proposition 1.1.16 (Continuity from above and below).
Limits of intersections and unions for certain sequences
(Ω, F, P)
Let Aₙ, Bₙ ∈ F, for n ∈ N such that
Aₙ ⊂ Aₙ₊₁ and Bₙ₊₁ ⊂ Bₙ for all n ∈ N.
Then,
i) P(∪ₙ Aₙ) = lim_{n→∞} P(Aₙ)
Sequence of events which is increasing (largerset cont original set)
ii) P(∩ₙ Bₙ) = lim{n→∞} P(Bₙ)
Sequence of events which is decreasing (smaller and smaller subsets of a set)
Proposition 1.1.16 (Continuity from above and below).
WHEN DOESNT IT WORK
When we don’t have decreasing increasing sequence
In general not true,
consider the real line
Ω,=[0,1]
P((a,b)) = b-a
B_1= [1/2,1]
B_n = [0,1] for all n >=2
then intersection of all B_n’s is [1/2,1] which has measure
1/2
but the limit of the final probability should be one (its one for all B_n where n =2,3,4,….
thus this doesnt work as don’t decrease
Exercise 1.1.17.
Consider (Ω, F, P)
Let A₁, A₂ be events such that
P(A₁) = P(A₂) = 1.
Show that P(A₁ ∩ A₂) = 1.
remark: A₁ and A₂ are not Ω; might have something smaller but not equal to Ω; there exists which have measure 0 without being the empty set e.g signleton set A={1/2} in measure [1/2 -e, 1/2 + e] = 2e will have measure 0}
(We know as A₁⊆ A₁∩A₂ andA₂ ⊆ A₁∩A₂ the probabilities will be ≤) proof will differ from notes
Sol:
Let us prove that P (A₁∪A₂)= P(A₁)+P(A₂)- P(A₁∩A₂)
We don’t know if they are disjoint so we express as disjoint union
A₁ ∪ A₂ = (A₁∩A₂) ∪ (A₁\A₂) ∪(A₂\A₁)
Thus by the properties of prob measures with this sum of disjoint sets
P(A₁∪A₂) = P(A₁∩A₂) +P(A₁\A₂)+P(A₂\A₁) (1)
Write A₁ = (A₁\A₂) ∪ (A₁∩A₂) disjoint
P(A₁) = P(A₁\A₂) + P(A₁∩A₂)
Rearranges to P(A₁\A₂) = P(A₁)- P(A₁∩A₂)
Write A₂ = (A₂\A₁) ∪ (A₁∩A₂) similarly rearranges to
P (A₂\A₁) = P(A₂ )- P(A₁∩A₂)
sub into (1)
P(A₁∪A₂) = P(A₁∩A₂) +[ P(A₁)- P(A₁∩A₂)]+[ P(A₂ )- P(A₁∩A₂)]
=P(A₁)+P(A₂)- P(A₁∩A₂)
Thus by using this formula if P(A₁) = P(A₂) = 1
We know
1 ≥P(A₁∪A₂) ≥ P(A₁) =1
thus P(A₁∪A₂)=1
P(A₁∪A₂) = 2- P(A₁∩A₂) (≤ 1 (its a probability he didnt use that fact) Thus 1≤ P(A₁∩A₂).
1 = 2- P(A₁∩A₂)
Thus P(A₁∩A₂) = 1
Exercise 1.1.18.
Let Aₙ be events such that P(Aₙ) = 1 for all n ∈ N.
Show that P(∩ₙ₌₁∞A) = 1.
HInt: Use Proposition 1.1.16
events such that each have probability 1 their infinite intersection will also:
Claim that ∀k∈N P( ∩ₙ₌₁ᵏ Aₙ) = 1
Base case: k=1 P( A₁ ) = 1
Induction hypothesis: Assume P( ∩ₙ₌₁ᵏ Aₙ) = 1
Inductive step: Consider
P( ∩ₙ₌₁ᵏ⁺¹ Aₙ) = P( (∩ₙ₌₁ᵏ Aₙ) ∩ Aₖ₊₁ ) We know that P(Aₖ₊₁ )=1 by given sequence and also by induction hypothesis we have that P( ∩ₙ₌₁ᵏ Aₙ) = 1
And the previous exercise shows
P( ∩ₙ₌₁ᵏ⁺¹ Aₙ) = P( (∩ₙ₌₁ᵏ Aₙ) ∩ Aₖ₊₁ ) = 1
Therefore we have shown by induction that this is true for all k in N.
Dealing with infinity:
Define new set Bₖ = ∩ₙ₌₁ᵏ Aₙ
B₁ =A₁
B₂ = A₁∩A₂ ⊆ A₁= B₁
B₃= A₁∩A₂∩A₃ ⊆ A₁∩A₂ = B₂
In general Bₖ₊₁ ⊆ Bₖ
decreasing seq of sets
we know that for a decreasing seq the probability of the intersection is the limit of the probabilities
∩ₙ₌₁∞ Bₖ = ∩ₙ₌₁∞ ∩ₙ₌₁ᵏ Aₙ = ∩ₙ₌₁∞ Aₙ
an intersection of decreasing sets thus
P(∩ₙ₌₁∞ Aₙ) = P(∩ₙ₌₁∞ Bₖ) = lim_{ₖ to ∞} P(Bₖ) = 1
we showed by induction so limit is 1
Explaining
{Aₖ infinitely open}
{Aₖ i.o.} = ∩ₙ₌₁ ∞ ∪ⱼ₌ₙ∞ Aⱼ
e.g Aₙ ∪ Aₙ₊₁ ∪ Aₙ₊₂= Bₙ
then take ∩ₙ₌₁ ∞ Bₙ
Take w∈{Aₖ i.o.} ⟺ w∈ ∩ₙ₌₁ ∞ ∪ⱼ₌ₙ∞ Aⱼ
⟺ w∈ ∪ⱼ₌ₙ∞ Aⱼ for all n in N
so there exists m>n s.t w∈ A_m
w belongs to infinitely many A_n
consider A₁,A₂,A₃,A₄,A₅,A₆,A₇,…
Give me and n =4 say, i can find m >n=4 s.t w ∈A_m
but if this is true for every n then I can always find a later set s.t w belongs, true for any n meaning w belongs to infinitely many.
Consider a coin flipped infinitely many times
(Ω, F, P)
Ω
Ω={H,T} flip once sample space
Flip twice then sample space
Ω={(H,H), (H,T), (T,H) ,(T,T)}
NOTATION differs to {H,H}
thus
Ω_2 = { (a₁,a₂) | a_i∈{H,T}}
Ωₙ = { (aₙ)ₙ₌₁∞ | a_i∈{H,T}}
Ω = { (aᵢ)ᵢ₌₁∞ | aᵢ∈{H,T}}
countable as its the # of reals
Consider a coin flipped infinitely many times
(Ω, F, P)
Define event
Ω = { (aᵢ)ᵢ₌₁∞ | aᵢ∈{H,T}}
Lets consider set/event
Aₙ= I get H nth time
Aₙ={(a₁,a₂,a₃,..,aₙ₋₁Haₙ₊₁,…l) |aᵢ∈{H,T} for i ≠n}
e.g A₂= {(H,H), (T,H)} = {(a₁,a₂)| |a₁∈{H,T} a₂ =H} ???
e.g A₂= {(H,H), (T,H), (T,H,T), (H,H,H), (H,H,T), (T,H,H),….}
If I choose w ∈ Aₙ I know w= (a₁,a₂,a₃,..,aₙ₋₁H……..)
If we take w ∈ {Aₖ i.o.} ⟺ w∈ ∩ₙ₌₁ ∞ ∪ⱼ₌ₙ∞ Aⱼ
⟺ w∈ ∪ⱼ₌ₙ∞ Aⱼ for all n in N
so there exists m>n s.t w∈ A_m
w belongs to infinitely many A_n
ie for n=1 there is an m_1 s.t w∈ A_m_1
n=2 there is an m_2 > m_1+1 s.t w∈ A_m_2….
So our H will be at position m_1,m_2,… true for any m
thus we have infinitely many heads?
SUMMARY OF PROPERTIES OF A PROBABILITY SPACE
If we have a subset A of B then P(A)≤ P(B)
B increasing sequence then intersection:
i) P(∪ₙ Aₙ) = lim_{n→∞} P(Aₙ)
Sequence of events which is increasing (largerset cont original set)
ii) P(∩ₙ Bₙ) = lim{n→∞} P(Bₙ)
Sequence of events which is decreasing (smaller and smaller subsets of a set)
Consider {Aₖ i.o.}ᶜ
complement of having infinitely many heads
we will have finitely many heads
we have heads up to some point, but after that there are only tails
{Aₖ i.o.}ᶜ = (∩ₙ₌₁ ∞ ∪ₙ₌ₘ∞ Aₙ)ᶜ de Morgans twice gives
= ∪ₙ₌₁ ∞ (∩ₙ₌ₘ∞ (Aₘᶜ))
Taking w∈ ∪ₙ₌₁ ∞ (∩ₙ₌ₘ∞ (Aₘᶜ)) = {Aₘᶜ eventually}
thus there exists an n s.t w∈ ∩ₙ₌ₘ∞ (Aₘᶜ))
this means there exists n in N s.t for all m≥n w∈ Aₘᶜ
ie for all positions greater than n we have w not in A_m so the position coords are not heads ie they are all tails
summary
consider seq A_n
then {A_n i.o}
{A_n eventually}
then {Aₖ i.o.} = ∩ₙ₌₁ ∞ ∪ⱼ₌ₙ∞ Aⱼ
{Aₙ eventually}= ∪ₙ₌₁ ∞ (∩ₙ₌ₘ∞ (Aₘᶜ))
Lemma 1.1.19 (Borel-Cantelli)
Consider (Ω, F, P) Aₖ⊆ F k in N
Let (Aₖ)ₖ₌₁∞ be a sequence of events. (so they lie in our sigma algebra)
If sumₖ₌₁∞ P(Aₖ) < ∞, then
P(Aₖ i.o.) = 0,
where
{Aₖ infinitely open}
{Aₖ i.o.} = ∩ₙ₌₁ ∞ ∪ⱼ₌ₙ∞ Aⱼ
If this is 0 then P({Aₖ i.o.}ᶜ)= P({ Aₖ eventually})= 1
the sum of the probabilities of the events {En} is finite
then the probability that infinitely many of them occur is 0, that is,
finish prev lecture
12 feb 1.2 expectations?
missed some?
Definition 1.1.20.
random variable
Let (Ω, F, P) probability space
function X:Ω → R is a **random variable*
if X is F -measurable (F/B(R)- measurable)
(meaning every time we take an element in B(R), then the pre-image of X is in F)
Exercise 1.1.21. Let X be a random variable on (Ω, F, P). Show that the collections of sets
{X−1(A) : A ∈ B(R)}
is a σ-algebra. In particular, it then follows that
{X−1(A) : A ∈ B(R)} = σ({X−1(A) : A ∈ B(R)}).
The σ-algebra above is called the σ-algebra generated by X and is denoted by σ(X).
notes
Remark 1.1.22. If we have two (or more) random variables X and Y then the σ-algebra generated
by X and Y is
σ(X, Y ) := σ({X−1(A) : A ∈ B(R)} ∪ {Y
−1(B) : B ∈ B(R)}).
Notice that the collection {X−1(A) : A ∈ B(R)} ∪ {Y−1(B) : B ∈ B(R)} is not a σ-algebra in
general(!).
In order to ease the notation, most of the times that we deal with a random variable X we
will drop the argument ω. For example, the set {ω ∈ Ω : X(ω) ≤ 1} will simply be denote by
{X ≤ 1} of the probability P({ω ∈ Ω : X(ω) ≤ 1}) will simply be denoted by P(X ≤ 1)
- X induces a probability measure Q_X on R
s.t Q_X: B(R) to [0,1]
Q_X is called the distribution of X or the law of X
Definition 1.1.23.
DISTRIBUTION OF X
Let (Ω, F, P) be a probability space and let X : Ω → R be a random variable.
The distribution of X is a probability measure Q_X on the (R, B(R)), which is given by
Q_X(A) = P(X ∈ A) for all A ∈ B(R) .
(=P(w∈Ω, X(w)∈A))
(if X is not measurable not defined)
elena side note:
Distribution function of a R.V X
F_X(x)= P( X≤ x)
=P(X in (-infinity,x])
is a particular case of Q_X
remember those sets generate the whole B(R)
Definition 1.2.1
random var is SIMPLE
A random variable X is called simple if there exist c_1, …, c_n ∈ R and events A_1, …, A_n ∈F (sigma algebra)
such that
X =Σᵢ₌₁ⁿ cᵢ1_A_ᵢ
Expectation of X
If X is a simple with representation defined prev, then we define the expectation of X by
E[X] : =Σᵢ₌₁ⁿ cᵢP(Aᵢ)
It turns out that the expectation does not depend on the representation of X
Expectation is a linear function
if X, Y are simple random variables and a, b ∈ R,
then
E[aX + bY ] = aE[X] + bE[Y ].
Monotonicity of expectation
It is also easy to see that if X, Y are simple random variables eith X ≤ Y , then E[X] ≤ E[Y ].
If X is a non-negative random
variable, then there exists
(If using that X is a non-negative RV then there exists a sequence random variables X_n, n ∈ N simple RVs
such that
for all ω ∈ Ω, X_n(ω) ↑ X(ω), as n ↑ ∞. Indeed, we can set X_n = h_n(X), where h_n(x) = min{[x2^n]2^{−n}, 2^n} where [a] denotes the lower integer part of a ∈ R.
Using that if X is a non negative RV then we can find an increasing sequence X_n of simple RVs s,t for all w∈Ω
X_n (w) converges to ↑ X(w)
as n converges to ↑infinity
Take X_n= h_n(X) where H_n(X)= min {x2^n} 2^{-n}, 2^n}
(lower integer part?)
(idea in notes)
Def: Expectation of X for any non-neg Rv
E(X) =ᵈᵉᶠ = limₙ→ ∞ E(X_n)
where X_n is a sequence converging to X as n tends to infinity X_n is an increasing sequence
well defined as X_n is a non decreasing seq
Def: RV X:Ω to R is integrable if
E[X+] < ∞, E[X−] < ∞,
where
X+=max{X,0}
X-= max{-X,0}
equivalently X is integrable
IFF
if E{|X|}< ∞
REMARK EEXPECTATIONS
Expectations are well-defined
bc X+ X- and |X| are non-negative
L₁(Ω),
denote the set of all integrable random variables by L₁(Ω).
L₁(Ω)={all integrable r.vs}
={X:Ω→R r.vs s.t E(|X|)< ∞ }
AND
For X ∈ L₁(Ω),
we set
E[X] = E[X⁺] − E[X⁻]
Lemma 1.2.2.
Let X, Y ∈L₁(Ω),and a, b ∈ R. Then the following hold:
The linearity and the monotonicity of the expectation is preserved on L₁(Ω)
Let X, Y ∈L₁(Ω),and a, b ∈ R. Then the following hold:
- E[aX + bY ] = aE[X] + bE[Y ] linearity
- If X ≤ Y , then E[X] ≤ E[Y ]. monotonicity
E(X) can also be denoted as
E(X)=
Let X be a random variable. Then, for any non-negative B(R)-measurable g : R → R we have
E[X)] =∫_Ω X dP
or ∫_Ω X(w) dP(w)
y the integral of X with respect to the measure P
(P is a measure, lebesgue measure wrt to P)
Example expectations
If X is a continous RV with PDF fₓ(x)
Then what is E(X)
E(g(x)
E(X)= ∫_R xfₓ(x) .dx
E(g(X))= ∫_R g(x)fₓ(x) .dx
can you always find pdf?
For these cases we work with the law of distribution of the RV
(for discrete is sum and probability mass funct)
Theorem 1.2.3.
integral expectation for g
Let X be a random variable. Then, for any non-negative B(R)-measurable g : R → R we have
E[g(X)] =∫_R g(x) Qₓ dx.
where Qₓ is the distribution function (probably measure)
valid for functs g taking values with different signs as long as well defined g+ and g- using g=g+ -g-
if X has a density function f, then
E(g(X))
E(g(X))= ∫_R g(x)fₓ(x) .dx
Actually, this is not a definition but a consequence of Theorem 1.2.3, since, the expression “X has a density f” means exactly that the distribution of X satisfies
Qₓ(dx) = fₓ(x) dx.
||X||ₗₚ
= (E[|X|ᵖ])¹/ᵖ
For p ∈ [1, ∞)
(when p=1 this norm is the expectation of value X;
collection of all random values in L_p space forms a linear subspace, 0 is constant therefore measurable and this tells you how far from 0)
(links to integral expectation E.G p=2)
||X||_L∞
= inf{K ≥ 0 : P(|X| ≤ K) = 1}.
For p ∈ [1, ∞)
so taking the K for which our RV absolute Probability is within 1
For p ∈ [1, ∞)
denote Lₚ(Ω)
L_p spaces
For p ∈ [1, ∞) denote Lₚ(Ω)
the linear space of all random variables X : Ω → R such that
||X||ₗₚ < ∞.
s,t the L_p norm is finite
Lemma 1.2.4 (Markov’s inequality).
Let X be a random variable. Then, for any ε > 0 we have
P(|X| ≥ ε) ≤ (1/ε)E[|X|]
probability greater than epsilon is bounded by a bound which depends on expectation
Lemma 1.2.5 (Hölders inequality).
Let p, q ∈ [1, ∞] with p⁻¹ + q⁻¹ = 1.
Then for any X,Y rvs
||XY ||_L₁ ≤ ||X||_Lₚ ||Y ||_L_q
L₁ norm is bounded for the product
(generalisation of cauchys inequality)
(E[|Xy|]) ≤ (E[|X|ᵖ])¹/ᵖ , (E[|Y|ᵠ])¹/ᵠ where power of q)
Lemma 1.2.6 (Minkowski’s inequality or triangle inequality)
Let p ∈ [1, ∞]. For any random variables X and Y we have
||X + Y ||_Lp ≤ ||X||_Lp + ||Y ||_Lp
(L_p spaces are normed vector spaces)
.
Exercise 1.2.7. Show that if p ≥ q, then ||X||_Lq ≤ ||X||_Lp
Hint: using Holders inequality
Let p, q ∈ [1, ∞] with p⁻¹ + q⁻¹ = 1. Then for any X,Y rvs
||XY ||_L₁ ≤ ||X||_Lₚ ||Y ||_L_q
Then
||1.X ||_L₁ ≤ ||1||_Lₚ ||X ||_L_q
= (E[|1.X|]) ≤ (E[|1|ᵖ])¹/ᵖ . (E[|X|ᵠ])¹/ᵠ
Suppose p ≥ q
(ᵠ is ^q here)
||X||_Lq ᵠ
= E[|X|^q] by defn
= E[1 |X|^q] by holder’s
≤ E[1][E[(|X|ᵠ)ᵖ/ᵠ]) ᵠ/ᵖ
=
||X||_Lp ^q
Hence, raising both sides of the inequality to the 1/q
finishes the proof
Theorem 1.2.8 (Completeness of Lₚ (Ω)).
Let p ∈ [1, ∞]. Let X_n ∈ LLₚ (Ω), for n ∈ N such that
lim_{n,m→∞} ||X_n − X_m||Lₚ = 0. (Cauchy)
Then there exists X ∈ L(Ω), such that
lim_n→∞ ||Xn − X||Lₚ = 0.
(Every cauchy seq in the space converges to some value in the space)
1.3 Convergence of random variables
As we have seen, random variables are functions defined on a set Ω. As such, we have different
modes of convergence.
Definition 1.3.1 (convergence almost surely)
For a sequence of RVs
Let (Xₙ)n∈N, be a sequence of random variables.
We say that Xₙ converges to a random variable X almost surely if
P{ω ∈ Ω : lim{n→∞} Xₙ(ω) = X(ω)}= 1.
We usually write Xₙ → X almost surely
(This is a measurable set, we apply the probability measure to the measurable set)
Definition 1.3.2 (convergence in probability)
Let (Xₙ)_n∈N, be a sequence of random variables.
We say that Xₙ converges to a random variable X in probability if for any ε > 0
lim {ₙ→∞} {ω ∈ Ω : |Xₙ(ω) − X(ω)| ≥ ε}= 0.
We usually write Xₙ → X in probability.
(probability: no matter how arbitrary small epsilon the limit)
Definition 1.3.3 (convergence in L_p).
Let (Xₙ)_n∈N, be a sequence of random variables that belong
to ₚ(Ω).
We say that Xn converges to a random variable X ∈Lₚ(Ω) in Lₚ if
lim_{n→∞} ||Xₙ − X||_Lₚ = 0
We usually write Xₙ → X in Lₚ
Theorem 1.3.4.
Relationship between types of convergence
Let (Xₙ)_ₙ∈N, X be random variables. The following hold.
(i) If Xₙ → X almost surely, then Xₙ → X in probability.
(ii) If Xₙ → X in probability, then there exists a subsequence (X_nₖ)k∈N such that X_nₖ → X almost surely.
(iii) If Xₙ → X in Lp, then Xₙ→ X in probability.
Exercise 1.3.5. Show (iii) from Theorem 1.3.4
(iii) If Xₙ → X in L_p, then Xₙ→ X in probability
Let ε > 0. By Markov’s inequality, we have
P(|Xₙ − X| > ε) = P(|Xₙ − X|ᵖ> εᵖ)
≤ (1/εᵖ) E[|Xₙ − X|ᵖ]
= (1/εᵖ) ||Xₙ − X||ₗₚᵖ
Taking limits: The right hand side of the above inequality tends to zero as n → ∞, by assumption. Hence, so
does the left hand side. This shows that X_n → X in probability
ignore:
Let (Xₙ)n∈N, be a sequence of random variables that belong
to ₚ(Ω).
Suppose Xₙ → X in Lₚ
for X ∈Lₚ(Ω) in Lₚ
THEN
lim{n→∞} ||Xₙ − X||_Lₚ = 0
by defn of convergence
for all ε > 0 there is an N s.t for all n>N
||Xₙ − X||_Lₚ < ε
thus…
we need to show that for any ε > 0
lim {ₙ→∞} {ω ∈ Ω : |Xₙ(ω) − X(ω)| ≥ ε}= 0.
Theorem 1.4.1 (Monotone Convergence).
Let Xₙ, X : Ω → R, for n ∈ N, be non-negative
random variables such that almost surely Xₙ ↑ X as n ↑ ∞. Then
E[X] = limn→∞ E[Xₙ]
Lemma 1.4.2 (Fatou’s Lemma)
Let Xₙ : Ω → R, for n ∈ N, be non-negative random variables.
Then
E[lim inf n→∞ Xₙ] ≤ lim_n→∞ inf E[Xₙ]
Theorem 1.4.3 (Lebesgue’s theorem on Dominated Convergence).
Suppose that Xₙ → X almost surely
and that there exists Y ∈ L₁(Ω)
such that |Xₙ| ≤ Y . (bounded)
Then X ∈ L_1(Ω) and
E[X] = lim_n→∞ E[Xₙ].
Definition 1.5.1. The events A1, …, An are called independent if
consider a probability space (Ω, F, P)
The events A₁, …, An are called independent if
P(∩ ᵢ₌₁ ⁿ Aᵢ₌₁) =
∏ᵢ₌₁ ⁿ P(A ᵢ).
probability is multiplies for intersections of indep events
sigma algebras G_1, …, G_n are called independent if
sigma algebras G_1, …, G_n are called independent if
for any A_1 ∈ G1, …, A_n ∈ G_n, the events
A_1, …, A_n are independent.
Random vars X_1, …X_n are independent if
The random variables X_1, …, X_n are independent if the σ-algebras σ(X_1), …σ(X_n) are independent
the elements of a family of events, σ-algebras, random
variables are independent the element of any finite sub-family are independent.
the elements of a family of events, σ-algebras, random
variables are independent if
the element of any finite sub-family are independent.
Lemma 1.5.2.
independence
Let G_1, G_2 ⊂ F be σ-algebras.
Let curly_A_1 ⊂ G_1 and curly_A_2 ⊂ G_2 be π-systems
(classes that are closed under finite intersection)
such that
σ(curly_A_1) = G_1 and σ(curly_A_2) = G_2. (sigma algebras gen by these events)
Then G_1 and G_2 are independent
if and only if
P(A_1 ∩ A_2) = P(A_1)P(A_2)
for all
A_1 ∈ curly_A1,
A2 ∈ curly_A2.
Theorem 1.5.3. If X, Y ∈ L_1(Ω) are independent random variables then
E(XY)
XY ∈ L_1(Ω) and
E[XY ] = E[X]E[Y ]
Lemma 1.5.4.
independence and measurable functs
Let X₁, …, Xₙ be independent random variables,
m₁ + … + mₖ = n
and
f₁, …, fₖ
be measurable functions of
Rᵐ₁, …, R ᵐₖ respectively.
Then the random variables
Y₁ = f₁(X₁, …, Xₘ₁),
Y₂ = f₂(Xₘ_₁ ₊₁ , …, Xₘ_₁ ₊ₘ_₂ ),
…,
Yk = f(Xₘ_₁+…+ₘ_{ₖ−₁}₊₁, …, Xₙ) are
independent.
Definition 1.5.5
sub algebra existence with conditional expectation
Let X ∈ L_1(Ω) and let G ⊂ F be a sub-σ-algebra of F. One can show that there exists a unique Y ∈ L1(Ω) with the following two properties:
- Y is G-measurable
- E[1_A X] = E[1_A Y ] for all A ∈ G
(indicator funct for A)
Y ~ conditional expectation of X given G
Y = E[X|G].
conditional expectation recap
For (Ω, F, P) and RV X
X:Ω to R
Let X ∈ L_1(Ω) ( IFF E[|x|] < infinity)
and let 𝒢 ⊂ F be a sub-σ-algebra of F.
There exists a unique RV Z∈ L_1(Ω) (z dep on Z and 𝒢)
s,t
E[1_A X] = E1_A Z]
for all A ∈𝒢
So conditional expectations given a sigma algebra are a random variable themselves
Z= E[X|𝒢]
Exercise 1.5.6. Suppose that Ω = ∪ᵢ₌₁ ⁿ Aᵢ where Aᵢ are disjoint events with P(Aᵢ) > 0. (DISJOINT PARTITION of sample space)
Let
𝒢 = σ({A₁, …., Aₙ}) (sigma algebra generated by a collection of subsets of Ω)
What do elements look like?
elements of sigma algebra 𝒢
Aᵢ∈𝒢
unions
∪_{i in I} Aᵢ ∈𝒢
complements
e.g complement of A_1 will be a union of other partition elements
intersections and complements of these turn out to be unions
turns out all elments are unions of the A_is
Exercise 1.5.6. Suppose that Ω = ∪ᵢ₌₁ ⁿ Aᵢ where Aᵢ are disjoint events with P(Aᵢ) > 0. (DISJOINT PARTITION of sample space)
Let
𝒢 = σ({A₁, …., Aₙ}) (sigma algebra generated by a collection of subsets of Ω)
show that E[X|𝒢] is a simple random variable given by
E[X|𝒢] = Σᵢ₌₁ ⁿ cᵢ 1_Aᵢ
where the constants cᵢ are given by
cᵢ = (1/P(Aᵢ)) E[1_Aᵢ X].
(It follows from the above that if 𝒢= {∅, Ω} is the trivial σ-algebra, then E[X|𝒢] = E[X]. We list below some properties of conditional expectations
We want to show that almost surely on Ω, we have E(X|𝒢) = Y , i.e. that
E[1_G X] = E[1_G Y ] ∀G ∈ 𝒢
We want to show that for all A∈𝒢
E[ 1_A x]= E[ 1_A Σᵢ₌₁ ⁿ cᵢ 1_Aᵢ]
Suppose A= Aⱼ j∈{1,…,n}
Then
E[ 1_Aⱼ Σᵢ₌₁ ⁿ cᵢ 1_Aᵢ]
=E[ Σᵢ₌₁ ⁿ cᵢ 1_Aᵢ 1_Aⱼ ] has value 1 only if we are considering the intersection, both 1
= E[ Σᵢ₌₁ ⁿ cᵢ 1_Aᵢ ∩_Aⱼ ] = E[cⱼ 1_Aⱼ ]
=()
——————————-
since A₁, . . . , Aₙ are disjoint, so
**1**Aᵢ ∩_Aⱼ =
{1∅ if i ≠ j (empty set has no elements)
{1Aᵢ if i = j
=
{0 if i ≠ j
{1_Aⱼ if i = j.
————————————
Replacing cⱼ
()=
= E[cⱼ 1_Aⱼ ]
= E[(1/P(Aⱼ)) E[1_Aⱼ X] 1_Aⱼ ]
= cⱼ E[1_Aⱼ ]
by linearity of expectation, multiplying by RV
=(1/P(Aⱼ))) E[1_Aⱼ X] E[1_Aⱼ]
by expectations of simple RVs (E[1_Aⱼ] =P(Aⱼ)
=E[1_Aⱼ X]
————-
Now checking the case for all A∈𝒢 :
Suppose A= ∪_{j∈I} Aⱼ j∈{1,…,n}
E[ 1∪{j∈I}Aⱼ Σᵢ₌₁ ⁿ cᵢ 1Aᵢ]
= E[ Σⱼ 1Aⱼ Σᵢ₌₁ ⁿ cᵢ 1_Aᵢ]
expectation of sum
=Σⱼ∈I E[1_Aⱼ Σᵢ₌₁ ⁿ cᵢ 1_Aᵢ]
singular A_j
=Σⱼ∈I E[1_Aⱼ Σᵢ₌₁ ⁿ cᵢ 1_Aᵢ]
=Σⱼ∈I E[1_Aⱼ X]
=E[1{∪{j∈I}Aⱼ} C ]
characteristic of a union
1∪{j∈I} =Σⱼ 1_Aⱼ
——-
suppose that Ω = ∪ᵢ₌₁ ⁿ Aᵢ where Aᵢ are disjoint events with P(Aᵢ) > 0. (DISJOINT PARTITION of sample space)
since A₁, . . . , Aₙ are disjoint, so
1_Aᵢ ∩_Aⱼ =
since A₁, . . . , Aₙ are disjoint, so
1_Aᵢ ∩Aⱼ =
{1∅ if i ≠ j (empty set has no elements)
{1Aᵢ if i = j
=
{0 if i ≠ j
{1_Aⱼ if i = j.
E[ Σᵢ₌₁ ⁿ 1_Aᵢ] for Aᵢ disjoint
“this is primary school question”
E[ Σᵢ₌₁ ⁿ cᵢ 1_Aᵢ] for Aᵢ disjoint
E[ Σᵢ₌₁ ⁿ 1_Aᵢ]
=Σᵢ₌₁ ⁿ P(Aᵢ)
E[ Σᵢ₌₁ ⁿ cᵢ 1_Aᵢ]
=Σᵢ₌₁ ⁿ cᵢ P(Aᵢ)
takes finitely many values
characteristic of a union
1∪{j∈I}Aⱼ
disjoint sets
suppose that Ω = ∪ᵢ₌₁ ⁿ Aᵢ where Aᵢ are disjoint events with P(Aᵢ) > 0. (DISJOINT PARTITION of sample space)
1∪{j∈I}Aⱼ
=Σⱼ 1_Aⱼ
=
beacuse w belongs to exactly one of them, disjoint thus others are 0
in general are conditional expectations RVS?
In general conditional expectations are
random variables not #
they can be numbers if sigma algebra is trivial
Proposition 1.5.7. Let X, Y ∈ L1(Ω) and G ⊂ F be a σ-algebra. The following hold
properties of conditional exp
Let X, Y ∈ L₁(Ω) (by defn can take the cond exp)
and 𝒢 ⊂ F be a σ-algebra. (by defn its a RV so is F-measurable)
The following hold
1. Linearity: If a, b are constants, then
E[aX + bY |𝒢] = aE[X|𝒢] + bE[Y |𝒢].
- Taking out what is known: If X is 𝒢-measurable and XY is integrable, then
E[XY |𝒢] = XE[Y |𝒢].
In particular, if X is G-measurable, then E[X|𝒢] = X. -
Conditioning on independent information: If W ⊂ F is a σ-algebra independent of
σ(σ(X), 𝒢), then E[X|σ(G, W)] = E[X|𝒢].
In particular, if σ(X) is independent of W,
then E[X|W] = E[X].
- Tower property: If W ⊂ 𝒢 is a σ-algebra, then we have E[X|W] = E[E[X|𝒢]|W].
- Monotonicity: If X ≥ 0 then E[X|𝒢] ≥ 0.
-
Conditional Jensen’s inequality: If φ : R → R is convex, such that φ(X) ∈ L₁(Ω),
then
φ(E[X|𝒢]) ≤ E[φ(X)|𝒢].
Exercise 1.5.8. Show that the conditional expectation is a continuous operator from L₁(Ω) to
L₁(Ω). In other words, show that if Xₙ, X ∈ L₁(Ω) such that lim_n→∞ ||Xₙ − X||_L₁ = 0,
then
lim_{n→∞} ||E[Xₙ|𝒢] − E[X|𝒢]||_L₁ = 0.
sequence of RVS converge in L₁
then conditional exp converges in L₁
Xₙ converges to X
E[Xₙ|G] converges to E[X|G]|
————-
Let (Ω, F, P) be a probability space, and let 𝒢 ⊂ F be a sub-σ-algebra. Suppose that Xₙ, X ∈ L_1(Ω, F, P) and that limn→∞ ||Xₙ − X||_L₁Ω = 0. Then
lim_n→∞
||E[Xₙ|𝒢] − E[X|𝒢]||L₁(Ω) =
as E[·|𝒢] is a linear operator:
=lim_n→∞
||E[Xₙ-X|𝒢]||_L₁Ω
By L_1-norm abs value
=lim_n→∞
E[ | E[Xₙ-X|𝒢] | ]
since | · | is convex, by Jensen’s inequality:
(absolute value inside)
≤ lim_n→∞
E[ E[ | Xₙ-X | |𝒢] ]
by the law of total expectation(or tower property)
lim_n→∞
E[ | Xₙ-X | ]
by the def. of the L1(Ω)-norm:
lim_n→∞
|| Xₙ-X ||L₁(Ω)
by assumption
=0
note that
E[ X |𝒢]
E[ X |𝒢]
=E[X]
————
suppose we take a trivial sigma algebra W= {Ø,Ω}
Then E[ X |W]
=E[X]
It suffices to check that
E[1_A E[X]]=
E[1_AX] for all A ∈W sigma algebra W
if this is true then LHS = E[ X |W]
is true as can either by empty set or Ω easy to check for 2
tower property
(E[E[X |𝒢] = E[ E[X |𝒢] |W]
= E[ X |W]= E[X] , as the trivial sigma algebra is always contained in 𝒢
Exercise 1.5.9. Let X ∈ L₂(Ω) (meaning ||X||_L₂ < infinity)
and let 𝒢 ⊂ F be a sub-σ-algebra of F.
Show that
||X − E[X|𝒢]||_L₂ =
min_Z∈L₂ᴳ ||X − Z||_L₂
where we have denoted by L₂ᴳ the collections of all 𝒢-measurable random variables that belong in
L₂(Ω).
L₂ᴳ subspace of
L₂(Ω).
By defn the conditional expectation E[X|G] is G-measurable
E[X|G] ∈L₂ᴳ(Ω).
(check: E[X|G] is G-measurable and finite uder L2 norm)
Moreover, by the conditional Jensen’s inequality we see that
E[|E[X|G]|²] ≤ E[E[|X|²|G]]
= E[|X|²] < ∞.
Consequently, E[X|G] ∈ L₂ᴳ and by definition of the minimum we get
||X − E[X|G]||L₂
≥ min{Z∈L₂ᴳ} ||X − Z||_L₂
(by definition I am taking the smallest from the set, for each RV Z i have a real # each one is above min)
tbc
.
prove equality by proving inequality in both directions
||X − E[X|𝒢]||_L₂ =
min_Z∈L₂ᴳ ||X − Z||_L₂
proving
||X − E[X|G]||L₂
≤ min{Z∈L₂ᴳ} ||X − Z||_L₂
part
On the other hand, for any Z ∈ L₂ᴳ
||X - Z||ₗ₂ ²
= ||X - Z − E[X|G] + E[X|G] ||ₗ₂ ²
By expanding ²
= ||X − E[X|G]||ₗ₂ ² + E[(X − E[X|G])(E[X|G] − Z)] +||X - Z||ₗ₂ ²
≥ ||X − E[X|G]||ₗ₂² + E[(X − E[X|G])(E[X|G] − Z)]
We claim the last term on the RHS is zero.
Indeed, since (E[X|G] − Z) and E[X|G] are G-measurable, we have
E[(X − E[X|G])(E[X|G] − Z)]=
we can take an extra conditional expectation inside the expectation bc the expectation of the conditional expectation is the expection itself
= E[E[(X − E[X|G])(E[X|G] − Z)|G]]
“(E[X|G] − Z)is G measurable something times something g-measurable given g we can use the property to pull it out”
= E[(E[X|G] − Z)E[(X − E[X|G])|G]]
“claim (E[X|G] − E[X|G])] is 0,as conditional exp are linear , by defn conditional exp is G measurable so cond exp wrt g again doesnt change”
= E[(E[X|G] − Z)(E[X|G] − E[X|G])]
= E[(E[X|G] − Z) · 0]
= 0.
showing for any such Z the inequality holds
L₂(Ω)=
L₂ᴳ (Ω)=
L₂ᴳ subspace of
L₂(Ω).
where we have denoted by L₂ᴳ the collections of all 𝒢-measurable random variables that belong in L₂(Ω).
L₂ᴳ (Ω)=
{X: Ω→R| X is 𝒢-measurable , ||X||_L₂ < ∞.}
L₂(Ω)=
{X: Ω→R| X is F-measurable , ||X||_L₂ < ∞.}
L₂ᴳ (Ω)⊆L₂(Ω)
this is because if I choose an X in L₂ᴳ (Ω) it is 𝒢-measurable
thus inverse images of borel sets belong to 𝒢:
∀A∈B(R), X⁻¹(A)∈𝒢⊆F
if I take any 𝒢-measurable it is automatically F-measurable due to 𝒢 ⊂ F being a sub-σ-algebra of F.
geometric interpretation
L₂(Ω)=
{X: Ω→R| X is F-measurable , ||X||_L₂ < ∞.}
consider R^2
plane taking vector in R^2
x,y in R^2
also operation +:closed under +
x+y also in R^2
closed under scalar multiplication:
ax in R ^2
if X is F-measurable , ||X||_L₂ < ∞:
im claiming that for two RVS X,Y
X+Y is also a RV in L_2
F-measurable
||X+Y||_L₂
by triangle
=<
||X||_L₂ +||Y||_L₂ < ∞
is also finite
aX is also a RV
||aX||_L₂ < a||X||_L₂ < ∞
another interpretation is thatL₂ᴳ (Ω)⊆L₂(Ω) lies on the line, any Z in L₂ᴳ (Ω) lies along the line, we can scale etc and maintain closure under these operations
scaling mean and variance
