1 Foundations of Probability Flashcards
Definition 1.1:
Sample space
The set Ω of all possible outcomes of an experiment
Definition 1.1:
Sample point
Each individual outcome ω of an experiment.
Sample points are elements of the sample space, denoted ω ∈ Ω.
Definition 1.1:
Types of sample spaces
Countable: Ω can be put into one-to-one correspondence with a subset of the positive integers Z+ = {1,2,3,…}. Sample points can be counted.
Uncountable: Not countable.
Definition 1.2:
Event
- A collection of possible outcomes of an experiment, i.e., a set of sample points.
- Event E is a subset of the sample space, E ⊂ Ω.
- E has occurred if ω ∈ E, i.e., outcome ω is in E.
Definition 1.3:
Subset
Event E is a subset of event F, denoted E ⊂ F, if F occurs whenever E occurs. i.e., if ω ∈ E ⇒ ω ∈ F.
Defintion 1.3:
Equal sets
Events E and F are equal, denoted E=F, iff E⊂F and F⊂E.
Definition 1.4:
Certain and impossible events, the empty set
- The sample space Ω must occur, thus termed the certain event.
- The empty set ∅ = { } is the subset of Ω which contains no outcomes. Event ∅ cannot occur, thus termed the impossible event.
Definition 1.5:
Elementary set operations - Union
Union of E and F:
* Event containing all outcomes that belong to either E or F or both.
* E ∪ F = {ω ∶ ω ∈ E or ω ∈ F }.
Definition 1.5:
Elementary set operations - Intersection
Intersection of E and F
* Event containing all outcomes that belong to both E and F.
* E ∩ F = {ω ∶ ω ∈ E and ω ∈ F }.
Definition 1.5:
Elementary set operations - Difference
Difference of E and F
* Event containing all outcomes that belong to E but not to F.
* E \ F = {ω ∶ ω ∈ E and ω ∉ F }.
Definition 1.5:
Elementary set operations - Complement
Complement of E
* Event containing all outcomes that do not belong to E.
* Eᶜ = {ω ∶ ω ∉ E}.
Definition 1.5:
Properties of complement
- (Eᶜ)ᶜ = E
- ∅ᶜ = Ω and Ωᶜ = ∅
- E ∪ Eᶜ = Ω
- E ∩ Eᶜ = ∅
Definition 1.6:
Disjoint events
Events E and F are disjoint or mutually exclusive if they cannot both occur at the same time, that is E∩F =∅.
Theorem 1.1:
Laws of set theory - Commutative laws
For any events E and F defined on a sample space Ω,
* E ∪ F = F ∪ E
* E ∩ F = F ∩ E
Theorem 1.1:
Laws of set theory - Associative laws
For any events E, F and G defined on a sample space Ω,
* E ∪ (F ∪ G) = (E ∪ F) ∪ G
* E ∩ (F ∩ G) = (E ∩ F) ∩ G
Theorem 1.1:
Laws of set theory - De Morgan’s laws
For any events E and F defined on a sample space Ω,
* (E ∪ F)ᶜ = Eᶜ ∩ Fᶜ
* (E ∩ F)ᶜ = Eᶜ ∪ Fᶜ
Theorem 1.1:
Laws of set theory - Distributive laws
For any events E, F and G defined on a sample space Ω,
* (E ∩ F) ∪ G = (E ∪ G) ∩ (F ∪ G)
* (E ∪ F) ∩ G = (E ∩ G) ∪ (F ∩ G)
Definition 1.7:
Event space
A collection F of subsets of Ω is called an event space or σ-algebra if it satisfies
1. ∅∈F
2. if E∈F, then Eᶜ∈F, (event space is closed under complementation).
3. if E₁ , E₂ , . . . ∈ F, then ⋃ᵢ₌₁∞ Eᵢ ∈ F, (event space is closed under countable unions).
F is a collection of all the events we would like to consider.
Definition 1.8:
Power set
For any set S, the power set of S, denoted by P(S), is the set of all subsets of S, including ∅ and S itself.
Definition 1.9:
Kolmogorov’s axioms (A1)-(A2)
- P(E) ≥ 0 for all events E ∈ F (probability of every event is non-negative)
- P(Ω) = 1 (probability of a certain event is 1)
Definition 1.9:
Kolmogorov’s axioms (A3)
If E₁ , E₂ , . . . are disjoint so that Eᵢ ∈ F and Eᵢ ∩ Eⱼ = ∅ for all i, j ∈ Z+, i ≠ j, then:
P(⋃ᵢ₌₁∞ Eᵢ) = ∑ᵢ₌₁∞P(Eᵢ) = P(E₁) + P(E₂) + . . . .
P is countably additive.
If the Eᵢ’s are disjoint then the probability that one of them occurs is the sum of their individual probabilities.
Definition 1.9:
Probability measure
Let Ω be a sample space and F an event space defined on Ω. A probability measure P on (Ω,F) satisfies Kolmogorov’s axioms.
The triple (Ω,F,P) is called a probability space and it is a mathematical model of a random experiment.
Theorem 1.2:
The probability of the empty set is zero
For any (Ω, F , P), we have P(∅) = 0.
Theorem 1.3:
Finite additivity of P for disjoint events
For any (Ω, F, P), if E, F ∈ F with E ∩ F = ∅ then
P(E ∪ F) = P(E) + P(F)
Corollary 1.1:
Probability of complements
For any (Ω,F,P) and E∈F, we have
P(Eᶜ)=1−P(E).
Corollary 1.2”
Probabilities are between zero and one
For any (Ω,F,P) and E ∈ F, we have
0≤P(E)≤1, and thus P∶F →[0,1].
Corollary 1.3:
Partition rule
For any (Ω,F,P) and E,F ∈ F, we have
P(F) = P(F ∩ E) + P(F ∩ Eᶜ).
Corollary 1.4:
Inclusion-exclusion rule
For any (Ω,F,P) and E,F ∈ F, we have
P(E ∪ F) = P(E) + P(F) − P(E ∩ F).
Corollary 1.5:
Containment rule
For any (Ω,F,P) and E ⊂ F ∈ F , we have
P(F) = P(E) + P(F ∩ Eᶜ).
so that P(F ) ≥ P(E).
Theorem 1.4:
Specifying probabilities for a countable sample space
For any event E ∈ F, P(E) = ∑i∶ωi∈E (pᵢ) where:
* Ω = {ω₁,…,ωₙ} is a finite set.
* F is any event space on Ω.
* p₁,…,pₙ be non-negative numbers that sum to one.
* The sum over an empty set is defined to be zero.
Then P is a probability measure on (Ω,F).
Probability of event E = ∑ probabiliies pᵢ of all outcomes ω in E.
This remains true if Ω = {ω₁,ω₂,…} with corresponding non-negative numbers p₁,p₂,… which sum to one.
