1 Foundations of Probability

Question 1

Definition 1.1:
Sample space

Answer 1

The set Ω of all possible outcomes of an experiment

Question 2

Definition 1.1:
Sample point

Answer 2

Each individual outcome ω of an experiment.
Sample points are elements of the sample space, denoted ω ∈ Ω.

Question 3

Definition 1.1:
Types of sample spaces

Answer 3

Countable: Ω can be put into one-to-one correspondence with a subset of the positive integers Z+ = {1,2,3,…}. Sample points can be counted.
Uncountable: Not countable.

Question 4

Definition 1.2:
Event

Answer 4

• A collection of possible outcomes of an experiment, i.e., a set of sample points.
• Event E is a subset of the sample space, E ⊂ Ω.
• E has occurred if ω ∈ E, i.e., outcome ω is in E.

Question 5

Definition 1.3:
Subset

Answer 5

Event E is a subset of event F, denoted E ⊂ F, if F occurs whenever E occurs. i.e., if ω ∈ E ⇒ ω ∈ F.

Question 6

Defintion 1.3:
Equal sets

Answer 6

Events E and F are equal, denoted E=F, iff E⊂F and F⊂E.

Question 7

Definition 1.4:
Certain and impossible events, the empty set

Answer 7

• The sample space Ω must occur, thus termed the certain event.
• The empty set ∅ = { } is the subset of Ω which contains no outcomes. Event ∅ cannot occur, thus termed the impossible event.

Question 8

Definition 1.5:
Elementary set operations - Union

Answer 8

Union of E and F:
* Event containing all outcomes that belong to either E or F or both.
* E ∪ F = {ω ∶ ω ∈ E or ω ∈ F }.

Question 9

Definition 1.5:
Elementary set operations - Intersection

Answer 9

Intersection of E and F
* Event containing all outcomes that belong to both E and F.
* E ∩ F = {ω ∶ ω ∈ E and ω ∈ F }.

Question 10

Definition 1.5:
Elementary set operations - Difference

Answer 10

Difference of E and F
* Event containing all outcomes that belong to E but not to F.
* E \ F = {ω ∶ ω ∈ E and ω ∉ F }.

Question 11

Definition 1.5:
Elementary set operations - Complement

Answer 11

Complement of E
* Event containing all outcomes that do not belong to E.
* Eᶜ = {ω ∶ ω ∉ E}.

Question 12

Definition 1.5:
Properties of complement

Answer 12

• (Eᶜ)ᶜ = E
• ∅ᶜ = Ω and Ωᶜ = ∅
• E ∪ Eᶜ = Ω
• E ∩ Eᶜ = ∅

Question 13

Definition 1.6:
Disjoint events

Answer 13

Events E and F are disjoint or mutually exclusive if they cannot both occur at the same time, that is E∩F =∅.

Question 14

Theorem 1.1:
Laws of set theory - Commutative laws

Answer 14

For any events E and F defined on a sample space Ω,
* E ∪ F = F ∪ E
* E ∩ F = F ∩ E

Question 15

Theorem 1.1:
Laws of set theory - Associative laws

Answer 15

For any events E, F and G defined on a sample space Ω,
* E ∪ (F ∪ G) = (E ∪ F) ∪ G
* E ∩ (F ∩ G) = (E ∩ F) ∩ G

Question 16

Theorem 1.1:
Laws of set theory - De Morgan’s laws

Answer 16

For any events E and F defined on a sample space Ω,
* (E ∪ F)ᶜ = Eᶜ ∩ Fᶜ
* (E ∩ F)ᶜ = Eᶜ ∪ Fᶜ

Question 16

Theorem 1.1:
Laws of set theory - Distributive laws

Answer 16

For any events E, F and G defined on a sample space Ω,
* (E ∩ F) ∪ G = (E ∪ G) ∩ (F ∪ G)
* (E ∪ F) ∩ G = (E ∩ G) ∪ (F ∩ G)

Question 17

Definition 1.7:
Event space

Answer 17

A collection F of subsets of Ω is called an event space or σ-algebra if it satisfies
1. ∅∈F
2. if E∈F, then Eᶜ∈F, (event space is closed under complementation).
3. if E₁ , E₂ , . . . ∈ F, then ⋃ᵢ₌₁∞ Eᵢ ∈ F, (event space is closed under countable unions).

F is a collection of all the events we would like to consider.

Question 18

Definition 1.8:
Power set

Answer 18

For any set S, the power set of S, denoted by P(S), is the set of all subsets of S, including ∅ and S itself.

Question 19

Definition 1.9:
Kolmogorov’s axioms (A1)-(A2)

Answer 19

1. P(E) ≥ 0 for all events E ∈ F (probability of every event is non-negative)
2. P(Ω) = 1 (probability of a certain event is 1)

Question 20

Definition 1.9:
Kolmogorov’s axioms (A3)

Answer 20

If E₁ , E₂ , . . . are disjoint so that Eᵢ ∈ F and Eᵢ ∩ Eⱼ = ∅ for all i, j ∈ Z+, i ≠ j, then:
P(⋃ᵢ₌₁∞ Eᵢ) = ∑ᵢ₌₁∞P(Eᵢ) = P(E₁) + P(E₂) + . . . .

P is countably additive.

If the Eᵢ’s are disjoint then the probability that one of them occurs is the sum of their individual probabilities.

Question 21

Definition 1.9:
Probability measure

Answer 21

Let Ω be a sample space and F an event space defined on Ω. A probability measure P on (Ω,F) satisfies Kolmogorov’s axioms.

The triple (Ω,F,P) is called a probability space and it is a mathematical model of a random experiment.

Question 22

Theorem 1.2:
The probability of the empty set is zero

Answer 22

For any (Ω, F , P), we have P(∅) = 0.

Question 23

Theorem 1.3:
Finite additivity of P for disjoint events

Answer 23

For any (Ω, F, P), if E, F ∈ F with E ∩ F = ∅ then
P(E ∪ F) = P(E) + P(F)

Question 24

Corollary 1.1:
Probability of complements

Answer 24

For any (Ω,F,P) and E∈F, we have
P(Eᶜ)=1−P(E).

Question 25

Corollary 1.2”
Probabilities are between zero and one

Answer 25

For any (Ω,F,P) and E ∈ F, we have
0≤P(E)≤1, and thus P∶F →[0,1].

Question 26

Corollary 1.3:
Partition rule

Answer 26

For any (Ω,F,P) and E,F ∈ F, we have
P(F) = P(F ∩ E) + P(F ∩ Eᶜ).

Question 27

Corollary 1.4:
Inclusion-exclusion rule

Answer 27

For any (Ω,F,P) and E,F ∈ F, we have
P(E ∪ F) = P(E) + P(F) − P(E ∩ F).

Question 28

Corollary 1.5:
Containment rule

Answer 28

For any (Ω,F,P) and E ⊂ F ∈ F , we have
P(F) = P(E) + P(F ∩ Eᶜ).
so that P(F ) ≥ P(E).

Question 29

Theorem 1.4:
Specifying probabilities for a countable sample space

Answer 29

For any event E ∈ F, P(E) = ∑i∶ωi∈E (pᵢ) where:
* Ω = {ω₁,…,ωₙ} is a finite set.
* F is any event space on Ω.
* p₁,…,pₙ be non-negative numbers that sum to one.
* The sum over an empty set is defined to be zero.
Then P is a probability measure on (Ω,F).

Probability of event E = ∑ probabiliies pᵢ of all outcomes ω in E.

This remains true if Ω = {ω₁,ω₂,…} with corresponding non-negative numbers p₁,p₂,… which sum to one.