1. Introduction To Probability Theory

Question 1

Suppose that we are about to perform an experiment whose outcome is not
predictable in advance. However, while the outcome of the experiment will not
be known in advance, let us suppose that the set of all possible outcomes is known.
What is this set of all possible outcomes of an experiment called?

Answer 1

The sample space

of the experiment and is denoted by S.

Question 2

If the experiment consists of the flipping of a coin, then
S = {H, T} where H means that the outcome of the toss is a head and T that it is a tail.
If the experiment consists of rolling a die, then the sample space is
S = {1, 2, 3, 4, 5, 6}
where the outcome i means that i appeared on the die, i = 1, 2, 3, 4, 5, 6.
What is the sample space if the experiments consists of flipping two coins?

Answer 2

S = {(H, H),(H, T),(T, H),(T, T)}

Question 3

What are subsets of sample spaces called?

Answer 3

Any subset E of the sample space S is known as an event.

Question 4

When do we say that the event E occurs?

Answer 4

We say that the event E occurs when the outcome of the experiment lies in E.

Question 5

For any two events E and F of a sample space S, what do we define the new event E∪F
to consist of?

Answer 5

All outcomes that are either in E or in F or in both E and F. That is,
the event E ∪ F will occur if either E or F occurs.

Question 6

How is the

event E∪F often referred to as?

Answer 6

The union of the event E and the event F.

Question 7

For any two events E and F, we may also define the new event EF, sometimes
written E∩F. What do we define the new event E∩F
to consist of?

Answer 7

EF consists
of all outcomes which are both in E and in F. That is, the event EF will occur
only if both E and F occur.

Question 8

How is the event E∩F referred to as?

Answer 8

The intersection of E and F.

Question 9

In Example (1) if E = {H} and F = {T}, then the event EF would not consist of any outcomes and hence could not occur. To give such an event a name, What shall we refer to it as?

Answer 9

The null event and denote it by Ø. (That is, Ø refers to the event
consisting of no outcomes.)

Question 10

What does it mean if EF = Ø?

Answer 10

Then E and F are said to be mutually

exclusive, or disjoint.

Question 11

We also define unions and intersections of more than two events in a similar manner. If E1, E2, … are events, then How is the union of of the events En denoted?

Answer 11

∪∞n=1 En, is defined to be the event that consists of all outcomes that are in En for at least one value of n = 1, 2, … .

Question 12

We also define unions and intersections of more than two events in a similar manner. If E1, E2, … are events, then how is the intersection of the events En denoted?

Answer 12

∩∞n=1 En, is defined to be the event consisting of those outcomes that are in all of the events En, n = 1, 2, … .

Question 13

Finally, for any event E what do we define the new event Ec (Notation looks like E^c), referred to as the complement of E, to consist of?

Answer 13

All outcomes in the sample space S that are not in E. That is, Ec will occur if and only if E does not occur.

Question 14

What should we note about the complement of E (at least as regards ex. 4)?

Answer 14

Note that since the experiment must result in some outcome, it follows that S^c = Ø.

Question 15

Consider an experiment whose sample space is S. For each event E of the sample
space S, we assume that a number P(E) is defined and satisfies the which three
conditions?

Answer 15

(i) 0 ≤ P(E) ≤ 1.
(ii) P(S) = 1.
(iii) For any sequence of events E1, E2, … that are mutually exclusive, that is, events for which EnEm = Ø when n ≠ m, then
P(∪∞n=1 En) =
= ∑∞n=1 P(En)

Question 16

What do we refer to P(E) as?

Answer 16

The probability of the event E.

Question 17

We have chosen to give a rather formal definition of probabilities as being functions defined on the events of a sample space. However, it turns out that these probabilities have which nice intuitive property?

Answer 17

Namely, if our experiment is repeated over and over again then (with probability 1) the proportion of time
that event E occurs will just be P(E).

Question 18

We have chosen to give a rather formal definition of probabilities as being functions defined on the events of a sample space. However, it turns out that these probabilities have a nice intuitive property. Namely, if our experiment is repeated over and over again then (with probability 1) the proportion of time
that event E occurs will just be P(E). Explain the reasoning behind this.

Answer 18

Since the events E and Ec are always mutually exclusive and since E∪Ec = S we have by (ii) and (iii) that
1 = P(S) = P(E∪Ec) = P(E)+P(Ec)
or
P(Ec) = 1 − P(E)

Question 19

Since the events E and Ec are always mutually exclusive and since E∪Ec = S we have by (ii) and (iii) that
1 = P(S) = P(E∪Ec) = P(E)+P(Ec)
or
P(Ec) = 1 − P(E)
What does this mean in words?

Answer 19

That the probability that an event does not occur

is one minus the probability that it does occur.

Question 20

We shall now derive a formula for P(E ∪ F), the probability of all outcomes either in E or in F. To do so, consider P(E) + P(F), which is the probability of all outcomes in E plus the probability of all points in F. Since any outcome that is in both E and F will be counted twice in P(E) + P(F) and only once in P(E ∪ F), what must we have?

Answer 20

P(E) + P(F) = P(E∪F) + P(EF)
or equivalently
P(E∪F) = P(E) + P(F) − P(EF)

Question 21

Note that when E and F are mutually exclusive (that is, when EF = Ø), what does the equation
P(E∪F) = P(E) + P(F) − P(EF) state?

Answer 21

That
P(E∪F) = P(E) + P(F) − P(Ø)
= P(E) + P(F)

Question 22

It can be shown by induction that, for any n events E1, E2, E3, … , En,
P(E1∪E2∪···∪En) =
=∑i P(Ei) −∑i

Answer 22

That the probability of the union of n events equals the sum of the probabilities of these events taken one at a time minus the sum of the probabilities of these events taken two at a time plus the sum of the probabilities of these events taken three at a time, and so on.

Question 23

Suppose that we toss two dice and that each of the 36 possible outcomes is equally likely to occur and hence has probability 1
36 . Suppose that we observe that the first die is a four. Then, given this information, what is the probability that the sum of the two dice equals six? How can we reason to calculate this probability?

Answer 23

Given that the initial die is a four, it follows that there can be at most six possible outcomes of our experiment, namely, (4, 1), (4, 2), (4, 3), (4, 4), (4, 5), and (4, 6). Since each of these outcomes originally had the same probability of occurring, they should still have equal probabilities. That is, given that the first die is a four, then the (conditional) probability of each of the outcomes (4, 1), (4, 2), (4, 3), (4, 4), (4, 5), (4, 6) is 1/6 while the (conditional) probability of the other 30 points
in the sample space is 0. Hence, the desired probability will be 1/6.

Question 24

If we let E and F denote, respectively, the event that the sum of the dice is six and the event that the first die is a four, then the probability just obtained is called the conditional probability that E occurs given that F has occurred. How is this denoted?

Answer 24

P(E|F)

Question 25

A general formula for P(E|F) that is valid for all events E and F is derived in the
same manner as the preceding. Explain.

Answer 25

Namely, if the event F occurs, then in order for E to occur it is necessary for the actual occurrence to be a point in both E and in F, that is, it must be in EF. Now, because we know that F has occurred, it follows that F becomes our new sample space and hence the probability that the event EF occurs will equal the probability of EF relative to the probability of F. That is,
P(E|F) = P(EF)/P(F).

Question 26

What should we note about Equation 1.5

P(E|F) = P(EF)/P(F)?

Answer 26

That Equation (1.5) is only well defined when P(F) > 0 and hence P(E|F) is only defined when P(F) > 0.

Question 27

When are two events E and F are said to be independent?

Answer 27

If

P(EF) = P(E)P(F)

Question 28

Two events E and F are said to be independent if
P(EF) = P(E)P(F)
By Equation (1.5), what does this imply?

Answer 28

That E and F are independent if
P(E|F) = P(E)
(which also implies that P(F|E) = P(F)).

Question 29

In words,when are two events said to be independent?

Answer 29

That is, E and F are independent if knowledge that F has occurred does not affect the probability that E occurs. That is, the occurrence of E is independent of whether or not F occurs.

Question 30

What are two events E and F that are not independent are said to be?

Answer 30

Dependent.