1 Discrete Probability Distributions

Question 1

In this chapter, we shall first consider chance experiments with a finite number of
possible outcomes ω1, ω2, . . . , ωn. For example, we roll a die and the possible
outcomes are 1, 2, 3, 4, 5, 6 corresponding to the side that turns up. We toss a coin
with possible outcomes H (heads) and T (tails).
It is frequently useful to be able to refer to an outcome of an experiment. For
example, we might want to write the mathematical expression which gives the sum
of four rolls of a die. How could we do this?

Answer 1

To do this, we could let Xi, i = 1, 2, 3, 4, represent the values
of the outcomes of the four rolls, and then we could write the expression
X1 + X2 + X3 + X4
for the sum of the four rolls.

Question 2

In the expression X1 + X2 + X3 + X4 for the sum of four rolls of the dice.
What are the Xi’s called

Answer 2

Random Variables

Question 3

What is a random variable?

Answer 3

A random variable
is simply an expression whose value is the outcome of a particular experiment.
Just as in the case of other types of variables in mathematics, random variables can
take on different values.

Question 4

Let X be the random variable which represents the roll of one die. How shall we
assign probabilities to the possible outcomes of this experiment.?

Answer 4

We do this by
assigning to each outcome ωj a nonnegative number m(ωj ) in such a way that
m(ω1) + m(ω2) + · · · + m(ω6) = 1

Question 5

What is the
m(ω1) + m(ω2) + · · · + m(ω6) = 1
function called?

Answer 5

The function m(ωj ) is called the distribution function of the random variable X.

Question 6

How should we assign probabilities for our dice example?

Answer 6

We would assign equal probabilities or probabilities
1/6 to each of the outcomes. With this assignment of probabilities, one could write
P(X ≤ 4) = 2/3
to mean that the probability is 2/3 that a roll of a die will have a value which does
not exceed 4.

Question 7

In both experiments (dice and coin), each outcome is assigned an equal probability.
This would certainly not be the case in general. For example, if a drug is found to
be effective 30 percent of the time it is used, we might assign a probability .3 that
the drug is effective the next time it is used and .7 that it is not effective. What does this last
example illustrate?

Answer 7

The intuitive frequency concept of probability. That is, if we have
a probability p that an experiment will result in outcome A, then if we repeat this
experiment a large number of times we should expect that the fraction of times that
A will occur is about p.

Question 8

We want to be able to perform an experiment that corresponds to a given set of
probabilities; for example, m(ω1) = 1/2, m(ω2) = 1/3, and m(ω3) = 1/6. How could we imagine an experiment to test this.

Answer 8

In this
case, one could mark three faces of a six-sided die with an ω1, two faces with an ω2,
and one face with an ω3.

Question 9

How could we visualize probabilities in a general case?

Answer 9

In the general case we assume that m(ω1), m(ω2), . . . , m(ωn) are all rational
numbers, with least common denominator n. If n > 2, we can imagine a long
cylindrical die with a cross-section that is a regular n-gon. If m(ωj ) = nj/n, then
we can label nj of the long faces of the cylinder with an ωj , and if one of the end
faces comes up, we can just roll the die again. If n = 2, a coin could be used to
perform the experiment.

Question 10

We will be particularly interested in repeating a chance experiment a large number
of times. Although the cylindrical die would be a convenient way to carry out
a few repetitions, why wouldn’t it be ideal?

Answer 10

It would be difficult to carry out a large number of experiments.
Since the modern computer can do a large number of operations in a very short
time, it is natural to turn to the computer for this task.

Question 11

What is a computer analog of rolling a die.

Answer 11

This is done on the computer

by means of a random number generator.

Question 12

We must first find a computer analog of rolling a die. This is done on the computer
by means of a random number generator. Depending upon the particular software
package, the computer can be asked for a real number between 0 and 1, or an integer
in a given set of consecutive integers. How is this done in the first case?

Answer 12

In the first case, the real numbers are chosen
in such a way that the probability that the number lies in any particular subinterval
of this unit interval is equal to the length of the subinterval.

Question 13

We must first find a computer analog of rolling a die. This is done on the computer
by means of a random number generator. Depending upon the particular software
package, the computer can be asked for a real number between 0 and 1, or an integer
in a given set of consecutive integers. How is this done in the second case?

Answer 13

In the second case,

each integer has the same probability of being chosen.

Question 14

Let X be a random variable with distribution function m(ω), where ω is in the
set {ω1, ω2, ω3}, and m(ω1) = 1/2, m(ω2) = 1/3, and m(ω3) = 1/6. If our computer
package can return a random integer in the set {1, 2, …, 6}, what should we do?

Answer 14

If our computer
package can return a random integer in the set {1, 2, …, 6}, then we simply ask it
to do so, and make 1, 2, and 3 correspond to ω1, 4 and 5 correspond to ω2, and 6
correspond to ω3.

Question 15

Let X be a random variable with distribution function m(ω), where ω is in the
set {ω1, ω2, ω3}, and m(ω1) = 1/2, m(ω2) = 1/3, and m(ω3) = 1/6. If our computer
package can return a random real number r in the
interval (0, 1), what should we do?

Answer 15

If our computer package returns a random real number r in the
interval (0, 1), then the expression
⌊6r⌋ + 1
will be a random integer between 1 and 6. (The notation ⌊x⌋ means the greatest
integer not exceeding x, and is read “floor of x.”)

Question 16

How does the program Random Numbers work?

Answer 16

(Random Number Generation) The program RandomNumbers
generates n random real numbers in the interval [0, 1], where n is chosen by the
user.

Question 17

(Coin Tossing) As we have noted, our intuition suggests that the
probability of obtaining a head on a single toss of a coin is 1/2. What’s the first way we can have the
computer toss a coin?

Answer 17

We can ask it to pick a random real number in the interval
[0, 1] and test to see if this number is less than 1/2. If so, we shall call the outcome
heads; if not we call it tails.

Question 18

(Coin Tossing) As we have noted, our intuition suggests that the
probability of obtaining a head on a single toss of a coin is 1/2. What’s the second way we can have the
computer toss a coin?

Answer 18

We can ask the computer
to pick a random integer from the set {0, 1}. The program CoinTosses carries
out the experiment of tossing a coin n times.

Question 19

We notice that when we tossed the coin 10,000 times, the proportion of heads
was close to the “true value” .5 for obtaining a head when a coin is tossed. What is a mathematical
model for this experiment called?

Answer 19

Bernoulli Trials (see Chapter 3).

Question 20

What will the

Law of Large Numbers, which we shall study later (see Chapter 8), show about Bernoulli trials?

Answer 20

The
Law of Large Numbers, will show that
in the Bernoulli Trials model, the proportion of heads should be near .5, consistent
with our intuitive idea of the frequency interpretation of probability.

Question 21

Of course, how could our coin toss program could be easily modified?

Answer 21

Our program could be easily modified to simulate coins for which the
probability of a head is p, where p is a real number between 0 and 1.

Question 22

In the case of coin tossing, we already knew the probability of the event occurring
on each experiment. Where does the real power of simulation come from?

Answer 22

The ability to estimate

probabilities when they are not known ahead of time.

Question 23

What is true about simulated results?

Answer 23

Accurate

results by simulation require a large number of experiments.

Question 24

The previous simulation shows that it is
important to know how many trials we should simulate in order to expect a certain
degree of accuracy in our approximation. What shall we later see?

Answer 24

That in these types of
experiments, a rough rule of thumb is that, at least 95% of the time, the error does
not exceed the reciprocal of the square root of the number of trials.

Question 25

We shall
show in the next section that for the first bet, the probability that de Mere wins is
1 − (5/6)^4 = .518. How can one understand this calculation?

Answer 25

The probability that no 6 turns up on the first toss is (5/6). The probability that no 6 turns up on either of the first two tosses is (5/6)^2. Reasoning in the same way, the probability that no 6
turns up on any of the first four tosses is (5/6)^4. Thus, the probability of at least
one 6 in the first four tosses is 1 − (5/6)^4.

Question 26

How shall we
show in the next section that for the second bet, the probability that de Mere wins is better for 25 throws of the dice than for 24?

Answer 26

With 24 rolls, the probability that de Mere wins is 1 − (35/36)^24 = .491, and for 25 rolls it is 1 − (35/36)^25 = .506.

Question 27

(Horse Races) Four horses (Acorn, Balky, Chestnut, and Dolby)
have raced many times. It is estimated that Acorn wins 30 percent of the time,
Balky 40 percent of the time, Chestnut 20 percent of the time, and Dolby 10 percent
of the time.
How can we have our computer carry out one race?

Answer 27

Choose a random
number x. If x < .3 then we say that Acorn won. If .3 ≤ x < .7 then Balky wins.
If .7 ≤ x < .9 then Chestnut wins. Finally, if .9 ≤ x then Dolby wins.
The program HorseRace uses this method to simulate the outcomes of n races.

Question 28

By the early 1900s it was clear that a better way to generate random numbers
was needed. In 1927, L. H. C. Tippett published a list of 41,600 digits obtained by
selecting numbers haphazardly from census reports. In 1955, RAND Corporation
printed a table of 1,000,000 random numbers generated from electronic noise. The
advent of the high-speed computer raised the possibility of generating random numbers
directly on the computer, and in the late 1940s. How did John von Neumann suggest
that this be done?

Answer 28

Suppose that you want a random sequence of four-digit
numbers. Choose any four-digit number, say 6235, to start. Square this number
to obtain 38,875,225. For the second number choose the middle four digits of this
square (i.e., 8752). Do the same process starting with 8752 to get the third number,
and so forth.

Question 29

After von Neumann’s squaring method, more modern methods evolved to involve the concept of modular arithmetic. Describe modular arithmetic.

Answer 29

If a is an
integer and m is a positive integer, then by a(mod m) we mean the remainder
when a is divided by m. For example, 10(mod 4) = 2, 8 (mod 2) = 0, and so
forth.

Question 30

After von Neumann’s squaring method, more modern methods evolved to involve the concept of modular arithmetic. How can we generate a random sequence X0, X1, X2, . . . of numbers?

Answer 30

Choose a starting number X0 and then obtain the numbers Xn+1 from Xn by the formula
Xn+1 = (aXn + c) (mod m),
where a, c, and m are carefully chosen constants. The sequence X0, X1, X2, . . . is then a sequence of integers between 0 and m − 1. To obtain a sequence of real
numbers in [0, 1), we divide each Xj by m. The resulting sequence consists of rational numbers of the form j/m, where 0 ≤ j ≤ m − 1. Since m is usually a very large integer, we think of the numbers in the sequence as being random real numbers in [0, 1).

Question 31

For both von Neumann’s squaring method and the modular arithmetic technique the sequence of numbers is actually completely determined by the first number. Thus, there is nothing really random about these sequences. Why do we use then them?

Answer 31

They produce
numbers that behave very much as theory would predict for random experiments.
To obtain different sequences for different experiments the initial number X0 is chosen by some other
procedure that might involve, for example, the time of day.