ST102.2 - Probability Theory

Question 1

What notation does capital N and lowercase n equate to?

Answer 1

• Capital N equates to population size.

- Lowercase n equates to sample size.

Question 2

What are statistical inferences and the associated assumptions?

Answer 2

The statistical inference will allow us to say things like the following about the population.

• ‘A 95% confidence interval for the population proportion, π, of ‘Yes’ voters is (0.5083, 0.5717).’
• ‘The null hypothesis that π = 0.50, against the alternative hypothesis that π > 0.50, is rejected at the 5% significance level.’
• In short, the opinion poll gives statistically significant evidence that ‘Yes’ voters are in the majority among likely voters.

The inferential statements about the opinion poll rely on the following assumptions and results.
- Each response Xi is a realisation of a random variable from a Bernoulli distribution with probability parameter π.
- The responses X1, X2, . . . , Xn are independent of each other.
The sampling distribution of the sample mean (proportion) X¯ has expected value π and variance π(1 − π)/n.
- By use of the central limit theorem, the sampling distribution is approximately a normal distribution.

Question 3

What is each response Xi a realisation of?

Answer 3

Each response Xi is a realisation of a random variable from a Bernoulli distribution with probability parameter π

Question 4

What are X1, X2, . . . , Xn all?

Answer 4

The responses X1, X2, . . . , Xn are independent of each other.

Question 5

What is the expected value and variance of the sample mean?

Answer 5

The sampling distribution of the sample mean (proportion) X¯ has expected value π and variance π(1 − π)/n.

Question 6

What will the sampling distribution be if we use the central limit theorem?

Answer 6

By use of the central limit theorem, the sampling distribution is approximately a normal distribution.

Question 7

What is probability theory?

Answer 7

Probability theory is the branch of mathematics that deals with randomness.

• Values in a sample are variable. If we collected a different sample we would not observe exactly the same values again.
• Values in a sample are also random. We cannot predict the precise values which will be observed before we actually collect the sample.

Question 8

What is a basic definition of an experiment?

Answer 8

An experiment is a process that produces outcomes and which can have several different outcomes.

Question 9

What is a basic definition of an outcome of an experiment?

Answer 9

The outcome of the experiment: for example, rolling a 3. It may be notated by S or Omega or lowercase omega.

Question 10

What is the basic definition of sample space?

Answer 10

Sample space S: the set of all possible outcomes/elements which are under consideration, here {1, 2, 3, 4, 5, 6}.

Question 11

What is the basic definition of an event?

Answer 11

Event: any subset A of the sample space, for example, A = {4, 5, 6}.

Question 12

What is the basic definition of the probability of an event?

Answer 12

The probability of an event, P(A), will be defined as a function that assigns probabilities (real numbers) to events (sets).
- The probability of an outcome A of an experiment is the proportion (relative frequency) of trials in which A would be the outcome if the experiment was repeated a very large number of times under similar conditions.

Question 13

What is the definition of a set?

Answer 13

A set is a collection of elements (also known as ‘members’ of the set).

• EXAMPLE.
• A = {Amy, Bob, Sam}.
• B = {1, 2, 3, 4, 5}.
• C = {x | x is a prime number} = {2, 3, 5, 7, 11, . . .}.
• D = {x | x ≥ 0} (that is, the set of all non-negative real numbers).

Question 14

What notation is used to denote membership or an element of a set?

Answer 14

• x ∈ A means that object x is an element of set A.
• x ∈/ A means that object x is not an element of set A (with a cross through ∈ to demonstrate that it is not a member of that set).
• EXAMPLE. If A = {1, 2, 3, 4, 5}, then:
• 1 ∈ A and 2 ∈ A.
• 6 ∈/ A and 1.5 ∈/ A.

Question 15

What notation is used to denote an empty set?

Answer 15

The empty set, denoted ∅, is the set with no elements, i.e. x ∈ ∅ / is true for every object x, and x ∈ ∅ is not true for any object x.
- This is an impossible event, therefore P(∅) = 0. The opposite of this would be a certain event where P = 1.

Question 16

What are venn diagrams?

Answer 16

The familiar Venn diagrams help to visualise statements about set.

• The S (the box which encloses the circles is our sample space).
• The circles represent the individual events or sets.

Question 17

What is important to remember about venn diagrams when it comes to proofs?

Answer 17

However, Venn diagrams are not formal proofs of results in set theory, but they act as visual aid instead.

Question 18

What is meant by a subset?

Answer 18

A ⊂ B means that set A is a subset of set B, defined as: A ⊂ B when x ∈ A ⇒ x ∈ B.

• Hence A is a subset of B if every element of A is also an element of B.
• Venn diagram depicting a subset, where A ⊂ B: (set A is enclosed within set B).
• Two sets A and B are equal (A = B) if they have exactly the same elements. This implies that A ⊂ B and B ⊂ A.
• EXAMPLE. {1, 2, 3} ⊂ {1, 2, 3, 4}, because all elements appear in the larger set
• EXAMPLE. {1, 2, 5} /⊂ {1, 2, 3, 4}, because the element 5 does not appear in the larger set.

Question 19

What is meant by a union of sets?

Answer 19

Unions of sets (‘or’) {Inclusive}

• The union, denoted ∪, of two sets is: A ∪ B = {x | x ∈ A or x ∈ B}.
• That is, the set of those elements which belong to A or B (or both).
• A ∩ B: both A and B happen.
• EXAMPLE. If A = {1, 2, 3, 4}, B = {2, 3} and C = {4, 5, 6}, then:
• A ∪ B = {1, 2, 3, 4}
• A ∪ C = {1, 2, 3, 4, 5, 6}
• B ∪ C = {2, 3, 4, 5, 6}.

Question 20

What is meant by an intersection of sets?

Answer 20

Intersections of sets (‘and’)

• The intersection, denoted ∩, of two sets is: A ∩ B = {x | x ∈ A and x ∈ B}. (x is such that x is a member of A and x is a member of B).
• That is, the set of those elements which belong to both A and B.
• A ∪ B: either A or B happens (or both happen).
• EXAMPLE. If A = {1, 2, 3, 4}, B = {2, 3} and C = {4, 5, 6}, then:
• A ∩ B = {2, 3}
• A ∩ C = {4}
• B ∩ C = ∅.

Question 21

What can you use as a union and intersection operator as a union and intersection of many sets?

Answer 21

Both set operators can also be applied to more than two sets, such as A ∩ B ∩ C.
Concise notation for the unions and intersections of sets A1, A2, . . . , An is:

n
U Ai = A1 ∪ A2 ∪ · · · ∪ An
i=1

and

n
∩ Ai = A1 ∩ A2 ∩ · · · ∩ An.
i=1

These can also be used for an infinite number of sets, i.e. when n is replaced by ∞.

Question 22

What is referred to as the complement (‘not’) of something?

Answer 22

If S is referred to as the sample space.
- Then A ⊂ S for every set A we may consider. The complement of A with respect to S is:
A^c = {x | x ∈ S and x ∈/ A}.
- That is, the set of those elements of S that are not in A.
- Therefore it is all but A.
- P(A) = 1 - P(A^c). AND ALSO P(A^c) = 1 − P(A).
- A^c: A does not happen, i.e. something other than A happens.

Question 23

What are the 4 set operators and why are they significant?

Answer 23

In proofs and derivations about sets, you can use the following set operators without proof.

1. Commutativity
2. Associativity
3. Distributive laws
4. De Morgan’s laws

Question 24

What is commutativity?

Answer 24

The “Commutative Laws” say we can swap numbers over and still get the same answer
- EXAMPLE. A ∩ B = B ∩ A and A ∪ B = B ∪ A.

Question 25

What is associativity?

Answer 25

The “Associative Laws” say that it doesn’t matter how we group the numbers (i.e. which we calculate first) …
- EXAMPLE. A ∩ (B ∩ C) = (A ∩ B) ∩ C and A ∪ (B ∪ C) = (A ∪ B) ∪ C.

Question 26

What are distributive laws?

Answer 26

The “Distributive Law” says that multiplying a number by a group of numbers added together is the same as doing each multiplication separately.
- EXAMPLE. A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C)
and A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C).

Question 27

What are De Morgan’s laws?

Answer 27

For “De Morgan’s Laws” the complement of the union of two sets is equal to the intersection of their complements and the complement of the intersection of two sets is equal to the union of their complements.
EXAMPLE. (A ∩ B)^c = A^c ∪ B^c
and (A ∪ B)^c = A^c ∩ B^c

Question 28

What is the complement of an empty set?

Answer 28

∅^c = S

The complement of an empty set is the sample space

Question 29

What are some further properties of set operations?

Answer 29

• ∅ ⊂ A, A ⊂ A and A ⊂ S.
• A ∩ A = A and A ∪ A = A.
• A ∩ A^c = ∅ and A ∪ A^c = S.
• If B ⊂ A, A ∩ B = B and A ∪ B = A.
• A ∩ ∅ = ∅ and A ∪ ∅ = A.
• A ∩ S = A and A ∪ S = S.
• ∅ ∩ ∅ = ∅ and ∅ ∪ ∅ = ∅.

Question 30

What does disjoint mean?

Answer 30

It means the same as mutually exclusive i.e there is not overlap and they do not occur at the same time.

Question 31

What determines if two sets A and B are disjoint or mutually exclusive?

Answer 31

Two sets A and B are disjoint or mutually exclusive if:

A ∩ B = ∅ (this does not mean they are mutually exclusive if A ∩ B = 0 as this is to do with probability).

Question 32

What are pairwise disjoint?

Answer 32

Sets A1, A2, . . . , An are pairwise disjoint if all pairs of sets from them are disjoint, i.e. Ai ∩ Aj = ∅ for all I /= j.

Question 33

What is a partition?

Answer 33

A partition is a set of pairwise Mutually Exclusive and Collectively Exhaustive sets (MECE).
- The sets A1, A2, . . . , An form a partition of the set A if they are pairwise disjoint and if:
n
U Ai = A, that is, A1, A2, . . . , An are collectively exhaustive of A.
i=1

Question 34

EXAMPLE. Suppose that A ⊂ B. Show that A and B ∩ Ac

form a partition of B.

Answer 34

We have: A ∩ (B ∩ A^c) = (A ∩ A^c) ∩ B = ∅ ∩ B = ∅ ME
and:
A ∪ (B ∩ A^c) = (A ∪ B) ∩ (A ∪ A^c) = B ∩ S = B. CE
- Hence A and B ∩ Ac are mutually exclusive and collectively exhaustive of B (MECE), and so they form a partition of B.

Question 35

What is probability in terms of axioms?

Answer 35

‘Probability’ is formally defined as a function P(·) from subsets (events) of the sample space S onto real numbers. Such a function is a probability function if it satisfies the following axioms (‘self-evident truths’).

Question 36

What is an axiom?

Answer 36

Axioms are self-evident truths upon which other probability results may be obtained, we are not proving these axioms but rather they provide us with a framework to derive probability results. They are a statement or proposition on which an abstractly defined structure is based. THEY ARE INVIOLABLE - THEY ARE LIKE LAW.
- EXAMPLE.
- Axiom 1: P(A) ≥ 0 for all events A. [0,1]
- Axiom 2: P(S) = 1.
- Axiom 3: If events A1, A2, . . . are pairwise disjoint (i.e. Ai ∩ Aj = ∅ for all i /= j ), then:
∞ ∞
P (U Ai) = Σ P(Ai)
i=1 i=1

Question 37

What are the 3 axioms?

Answer 37

The axioms require that a probability function must always satisfy these requirements.
1. Axiom 1 requires that probabilities are always non-negative. P(A) ≥ 0 for all events A. [0,1]
2. Axiom 2 requires that the outcome is some element from the sample space with certainty (that is, with probability 1). In other words, the experiment must have some outcome. P(S) = 1. Therefore the sample space is = 1.
3. Axiom 3 states that if events A1, A2, . . . are mutually exclusive, the probability of their union is simply the sum of their individual probabilities.
- If events A1, A2, . . . are pairwise disjoint (i.e. Ai ∩ Aj = ∅ for all i /= j ), then:
∞ ∞
P (U Ai) = Σ P(Ai)
i=1 i=1

All other properties of the probability function can be derived from the axioms. We begin by showing that a result like Axiom 3 also holds for finite collections of mutually exclusive sets.

Question 38

What is the proof for empty sets using axioms?

Answer 38

For the empty set, ∅, we have: P(∅) = 0.
Proof: Since ∅ ∩ ∅ = ∅ and ∅ ∪ ∅ = ∅, Axiom 3 gives:
P(∅) = P(∅ ∪ ∅ ∪ · · ·) = ∞Σ(i=1)P(∅).
However, the only real number for P(∅) which satisfies this is P(∅) = 0.

Question 39

How can we proove that the sum of a pairwise disjoint past n + 1 is an empty set?

Answer 39

If A1, A2, . . . , An are pairwise disjoint, then:
n n
P (U Ai) = Σ P(Ai)
i=1 i=1

Proof: In Axiom 3, set An+1 = An+2 = · · · = ∅, so that:
   ∞            ∞
P (U Ai)  =  Σ P(Ai)
   i=1           i=1
THEREFORE:
   n            ∞                n
Σ P(Ai)  +  Σ P(Ai)   =   Σ P(Ai)
   i=1           i=n + 1      i=1 

since P(Ai) = P(∅) = 0 for i = n + 1, n + 2, . . ..

Question 40

How can we sum probabilities of mutually exclusive sets?

Answer 40

P(A) = P(A1) + P(A2) + P(A3).

That is, we can simply sum probabilities of mutually exclusive sets. This is very useful for deriving further results.

Question 41

How can we use prove that the probability of the complement of A is equal to the probability of 1 - P(A)?

Answer 41

Proof: We have that A ∪ A^c = S and A ∩ A^c = ∅. MECE.
Therefore using axiom 2 as the probability of the LHS = RHS:
1 = P(S) = P(A ∪ A^c) = P(A) + P(A^c)
using the previous result with with n = 2, A1 = A and A2 = A^c

Question 42

How can you prove that probabilities are between 0 and 1?

Answer 42

For any event A, we have: P(A) ≤ 1.
Proof (by contradiction): If it was true that P(A) > 1 for some A, then we would have:
- P(A^c) = 1 − P(A) < 0.
- This violates Axiom 1, so cannot be true.
- Therefore, it must be that P(A) ≤ 1 for all A.
- Putting this and Axiom 1 together, we get:
0 ≤ P(A) ≤ 1 for all events A.

Question 43

How can you prove if A ⊂ B, then P(A) ≤ P(B)?

Answer 43

Proof: Partition B as:
B = A ∪ (B ∩ A^c) where the two sets in the union are disjoint.

Therefore:
- P(B) = P(A ∪ (B ∩ A^c)) = P(A) + P(B ∩ A^c) ≥ P(A) since P(B ∩ A^c) ≥ 0.

Question 44

How can you prove P(A ∪ B) = P(A) + P(B) − P(A ∩ B).

Answer 44

Proof: Using partitions (dividing into MECE):
- P(A ∪ B) = P(A ∩ B^c) + P(A ∩ B) + P(A^c ∩ B)
- P(A) = P(A ∩ B^c) + P(A ∩ B)
- P(B) = P(A^c ∩ B) + P(A ∩ B)
and hence:
- P(A ∪ B) = (P(A) − P(A ∩ B)) + P(A ∩ B) + (P(B) − P(A ∩ B)) = P(A) + P(B) − P(A ∩ B).

Question 45

What are the summaries of important probability functions?

Answer 45

In summary, the probability function has the following properties.

• P(S) = 1 and P(∅) = 0.
• 0 ≤ P(A) ≤ 1 for all events A.
• If A ⊂ B, then P(A) ≤ P(B).

These show that the probability function has the kinds of values we expect of something called a ‘probability’.

• P(A^c) = 1 − P(A).
• P(A ∪ B) = P(A) + P(B) − P(A ∩ B).

These are useful for deriving probabilities of new events.

Question 46

What is the difference between empirical probability and theoretical probability?

Answer 46

Experimental probability is the probability that is determined on the basis of the results of an experiment repeated many times.
- Calculated as the (Number of times occurred)/(Total number of times experiment performed).

Theoretical probability is the probability that is determined on the basis of reasoning.

Question 47

What is classical probability?

Answer 47

Classical probability is a simple special case where values of probabilities can be found by just counting outcomes. This requires that:

1. The sample space contains only a finite number of outcomes.
2. All of the outcomes are equally likely.

EXAMPLE of classical probability.

• Tossing a coin (heads or tails) one or more times.
• Rolling one or more dice (each scored 1, 2, 3, 4, 5 or 6).
• Drawing one or more playing cards from a deck of 52 cards.