intermediate probability

Question 1

conditional probability

Answer 1

how does the probability of the event changes if we have extra information?

The conditional probability of 𝐴 knowing that 𝐵 occurred is written
𝑃(𝐴|𝐵)

the conditional probability of 𝐴 given 𝐵

Question 2

formal definition of conditional probability

Answer 2

Question 3

multiplication rule

Answer 3

𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴|𝐵) ⋅ 𝑃(𝐵).

This is simply a rewriting of the definition in Equation (1) of conditional probability.

We will see that our use of the multiplication rule is very similar to our use of the rule of
product in counting.

Question 4

law of total probability

Answer 4

Suppose the sample space Ω is divided into 3 disjoint events 𝐵1, 𝐵2, 𝐵3 (see the figure
below). Then for any event A:

𝑃 (𝐴) = 𝑃 (𝐴 ∩ 𝐵1) + 𝑃 (𝐴 ∩ 𝐵2) + 𝑃 (𝐴 ∩ 𝐵3)

𝑃(𝐴) = 𝑃(𝐴|𝐵1) 𝑃(𝐵1) + 𝑃(𝐴|𝐵2) 𝑃(𝐵2) + 𝑃(𝐴|𝐵3) 𝑃(𝐵3)

Question 5

An urn contains 5 red balls and 2 green balls. Two balls are drawn one after
the other. What is the probability that the second ball is red?

Answer 5

The sample space is Ω = {rr, rg, gr, gg}.
Let 𝑅1 be the event ‘the first ball is red’, 𝐺1 = ‘first ball is green’, 𝑅2 = ‘second ball is
red’, 𝐺2 = ‘second ball is green’. We are asked to find 𝑃(𝑅2).

Let’s compute this same value using the law of total probability (3). First, we’ll find the conditional probabilities. This is a simple counting exercise.
𝑃(𝑅2|𝑅1) = 4/6, 𝑃(𝑅2|𝐺1) = 5/6.

Since 𝑅1 and 𝐺1 partition Ω the law of total probability says
𝑃(𝑅2) = 𝑃(𝑅2|𝑅1)𝑃(𝑅1) + 𝑃(𝑅2|𝐺1)𝑃(𝐺1) = 5/7

Question 6

probability urns

Answer 6

In probability and statistics, an urn problem is an idealized mental exercise
in which some objects of real interest (such as atoms, people, cars, etc.) are
represented as colored balls in an urn or other container. One pretends to draw (remove) one or more balls from the urn; the goal is to determine the probability of drawing one color or another, or some other properties. A key parameter is whether each ball is returned to the urn after each draw.

Question 7

An urn contains 5 red balls and 2 green balls. A ball is drawn. If it’s green
a red ball is added to the urn and if it’s red a green ball is added to the urn. (The original ball is not returned to the urn.) Then a second ball is drawn. What is the probability the second ball is red?

Answer 7

The law of total probability says that 𝑃(𝑅2) can be computed using the expression in Equation (4). Only the values for the probabilities will change. We have
𝑃(𝑅2|𝑅1) = 4/7, 𝑃(𝑅2|𝐺1) = 6/7.

Therefore, 𝑃(𝑅2) = 𝑃(𝑅2|𝑅1)𝑃(𝑅1) + 𝑃(𝑅2|𝐺1)𝑃(𝐺1) = (4/7)*(5/7) +(6/7)(2/7) = 32/49

Question 8

trees to organize computation

Answer 8

consider the case of the urn with 5 red balls and two green balls. if the first ball drawn is green, a red ball is added to the urn, and if the first ball drawn is red, a green ball is added to the urn. what is the probability of drawing a red ball second?

Question 9

probability tree

Answer 9

node - Each dot is called a node.
levels - The tree is organized by levels.

The top node (root node) is at level 0. The next layer down is level 1 and so on. Each level shows the outcomes at one stage of the game. Level 1 shows the possible outcomes of the first draw. Level 2 shows the possible outcomes of the second draw starting from each node in level 1. Probabilities are written along the branches. The probability of 𝑅1 (red on the first draw) is 5/7.

It is written along the branch from the root node to the one labeled 𝑅1. At the
next level we put in conditional probabilities. The probability along the branch from 𝑅1 to 𝑅2 is 𝑃(𝑅2|𝑅1) = 4/7. It represents the probability of going to node 𝑅2 given that you are already at 𝑅1.
The multiplication rule says that the probability of getting to any node is just the product of the probabilities along the path to get there. For example, the node labeled 𝑅2 at the far left really represents the event 𝑅1 ∩ 𝑅2 because it comes from the 𝑅1 node. The multiplication rule now says

𝑃(𝑅1 ∩ 𝑅2) = 𝑃 (𝑅1) ⋅ 𝑃 (𝑅2|𝑅1)

The law of total probability is just the statement that 𝑃(𝑅2) is the sum of the probabilities of all paths leading to 𝑅2 (the two circled nodes in the figure).

Question 10

independence

Answer 10

Two events are independent if knowledge that one occurred does not change the probability that the other occurred.

Informally, events are independent if they do not influence one another

Question 11

formulation of independence

Answer 11

𝑃(𝐴 ∩ 𝐵) = 𝑃 (𝐴) ⋅ 𝑃 (𝐵)

1. If 𝑃 (𝐵) ≠ 0 then 𝐴 and 𝐵 are independent if and only if 𝑃(𝐴|𝐵) = 𝑃(𝐴).
2. If 𝑃 (𝐴) ≠ 0 then 𝐴 and 𝐵 are independent if and only if 𝑃(𝐵|𝐴) = 𝑃(𝐵).

Question 12

Toss a fair coin twice. Let 𝐻1 = ‘heads on first toss’ and let 𝐻2 = ‘heads on
second toss’. Are 𝐻1 and 𝐻2 independent?

Answer 12

Since 𝐻1 ∩ 𝐻2 is the event ‘both tosses are heads’ we have

𝑃(𝐻1 ∩ 𝐻2) = 1/4 = 𝑃(𝐻1)𝑃(𝐻2).

Therefore the events are independent.

Question 13

Toss a fair coin 3 times. Let 𝐻1 = ‘heads on first toss’ and 𝐴 = ‘two heads
total’. Are 𝐻1 and 𝐴 independent?

Answer 13

We know that 𝑃(𝐴) = 3/8. Since this is not 0 we can check if the formula 𝑃(𝐴|𝐵) = 𝑃(𝐴) holds.

Now, 𝐻1 = {HHH, HHT, HTH, HTT} contains exactly two outcomes (𝐻𝐻𝑇 , 𝐻𝑇 𝐻) from 𝐴, so we have 𝑃(𝐴|𝐻1) = 2/4.

Since 𝑃(𝐴|𝐻1) ≠ 𝑃(𝐴) these events are not independent.

Question 14

Draw one card from a standard deck of playing cards. Let’s examine the
independence of 3 events ‘the card is an ace’, ‘the card is a heart’ and ‘the card is red’.

Define the events as 𝐴 = ‘ace’, 𝐻 = ‘hearts’, 𝑅 = ‘red’.

Answer 14

(a) We know that 𝑃(𝐴) = 4/52 (4 out of 52 cards are aces), 𝑃(𝐴|𝐻) = 1/13 (1 out of 13 hearts are aces). Since 𝑃 (𝐴) = 𝑃 (𝐴|𝐻) we have that 𝐴 is independent of 𝐻.

(b) 𝑃(𝐴|𝑅) = 2/26 = 1/13 = 𝑃(𝐴). So 𝐴 is independent of 𝑅. That is, whether the card is an ace is independent of whether it is red.

(c) Finally, what about 𝐻 and 𝑅? Since 𝑃 (𝐻) = 1/4 and 𝑃 (𝐻|𝑅) = 1/2, 𝐻 and 𝑅 are not independent. We could also see this the other way around: 𝑃(𝑅) = 1/2 and 𝑃 (𝑅|𝐻) = 1, so 𝐻 and 𝑅 are not independent. That is, the suit of a card is not independent of the color of the card’s suit.

Question 15

paradoxes of independence

Answer 15

An event 𝐴 with probability 0 is independent of itself, since in this case both sides of equation (6) are 0. This appears paradoxical because knowledge that 𝐴 occurred certainly gives information about whether 𝐴 occurred. We resolve the paradox by noting that since 𝑃(𝐴) = 0 the statement ‘𝐴 occurred’ is vacuous.

Think: For what other value(s) of 𝑃(𝐴) is 𝐴 independent of itself?

Question 16

baye’s theorem

Answer 16

For two events 𝐴 and 𝐵 Bayes’ theorem (also called Bayes’ rule and Bayes’
formula) say (reference the fig)

1. Bayes’ rule tells us how to ‘invert’ conditional probabilities, i.e. to find
   𝑃(𝐵|𝐴) from 𝑃(𝐴|𝐵).
2. In practice, 𝑃(𝐴) is often computed using the law of total probability.

Question 17

proof of baye’s theorem

Answer 17

The key point is that 𝐴 ∩ 𝐵 is symmetric in 𝐴 and 𝐵. So the multiplication rule says
𝑃(𝐵|𝐴) ⋅ 𝑃(𝐴) = 𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴|𝐵) ⋅ 𝑃(𝐵).
Now divide through by 𝑃(𝐴) to get Bayes’ rule.
A common mistake is to confuse the meanings of 𝑃(𝐴|𝐵) and 𝑃(𝐵|𝐴). They can be very
different. This is illustrated in the next example.

Question 18

Toss a coin 5 times. Let 𝐻1 = ‘first toss is heads’ and let 𝐻𝐴 = ‘all 5 tosses
are heads’. Then 𝑃(𝐻1|𝐻𝐴) = 1 but 𝑃(𝐻𝐴|𝐻1) = 1/16.

use Bayes’ theorem to compute 𝑃(𝐻1|𝐻𝐴) using 𝑃(𝐻𝐴|𝐻1)

Answer 18

The terms are 𝑃(𝐻𝐴|𝐻1) = 1/16, 𝑃(𝐻1) = 1/2, 𝑃(𝐻𝐴) = 1/32.

Question 19

base rate fallacy

Answer 19

The base rate fallacy is one of many examples showing that it’s easy to confuse the meaning of 𝑃(𝐵|𝐴) and 𝑃(𝐴|𝐵) when a situation is described in words. This is one of the key
examples from probability and it will inform much of our practice and interpretation of statistics. You should strive to understand it thoroughly.

Question 20

Consider a routine screening test for a disease. Suppose the frequency of the disease in the population (base rate) is 0.5%. The test is fairly accurate with a 5% false positive rate and a 10% false negative rate.

You take the test and it comes back positive. What is the probability that you have the disease?

Answer 20

𝐷+ = ‘you have the disease’
𝐷− = ‘you do not have the disease
𝑇 + = ‘you tested positive’
𝑇 − = ‘you tested negative’.

We are given 𝑃(𝐷+) = 0.005 and therefore 𝑃(𝐷−) = 0.995. The false positive and false
negative rates are (by definition) conditional probabilities.

𝑃(false positive) = 𝑃(𝑇 +|𝐷−) = 0.05 and 𝑃(false negative) = 𝑃(𝑇 −|𝐷+) = 0.1.

The complementary probabilities are known as the true negative and true positive rates:

𝑃(𝑇 −|𝐷−) = 1 − 𝑃(𝑇 +|𝐷−) = 0.95
𝑃(𝑇 +|𝐷+) = 1 − 𝑃(𝑇 −|𝐷+) = 0.9.

Using baye’s theorem on these probabilities to get 𝑃(𝐷+|𝑇 +) we calculate around 8.3%!!

Question 21

remarks on base rate fallacy

Answer 21

This is called the base rate fallacy because the base rate of the disease in the population is so low that the vast majority of the people taking the test are healthy, and even with an accurate test most of the positives will be healthy people. Ask your doctor
for his/her guess at the odds.

To summarize the base rate fallacy with specific numbers 95% of all tests are accurate does not imply 95% of positive tests are accurate

We will refer back to this example frequently. It and similar examples are at the heart of many statistical misunderstandings.

Question 22

toss a fair coin 3 times

what is the probability of 3 heads?

Answer 22

Sample space Ω = {𝐻𝐻𝐻, 𝐻𝐻𝑇 , 𝐻𝑇𝐻, 𝐻𝑇𝑇 , 𝑇𝐻𝐻, 𝑇𝐻𝑇 , 𝑇𝑇𝐻, 𝑇𝑇𝑇 }.

All outcomes are equally probable, so 𝑃 (3 heads) = 1/8.

Question 23

toss a fair coin 3 times

Suppose the first toss was heads

given this info how should we compute the probability of 3 heads?

Answer 23

Question 24

conditional probability

Answer 24

takes into account additional conditions

Question 25

the conditional probability of A knowing that B occurred is written

Answer 25

𝑃 (𝐴|𝐵)

‘the conditional probability of 𝐴 given 𝐵’

Question 26

Draw two cards from a deck. Define the events: 𝑆1 = ‘first card is a spade’ and 𝑆2 = ‘second card is a spade’. What is the 𝑃 (𝑆2|𝑆1)?

Answer 26

Refer to other cards where we compute 𝑃 (𝑆1), 𝑃 (𝑆2) and 𝑃 (𝑆1 ∩ 𝑆2). We use the formula above, and get

Question 27

Draw two cards from a deck. Define the events: 𝑆1 = ‘first card is a spade’ and 𝑆2 = ‘second card is a spade’.

What is 𝑃(𝑆1∩𝑆2)?

Answer 27

We compute 𝑃 (𝑆1 ∩ 𝑆2) by counting:

Number of ways to draw a spade followed by a second spade: 13 ⋅ 12.

Number of ways to draw any card followed by any other card: 52 ⋅ 51.

Question 28

Draw two cards from a deck. Define the events: 𝑆1 = ‘first card is a spade’ and 𝑆2 = ‘second card is a spade’. What is the 𝑃 (𝑆2)?

Answer 28

Since 13 of the 52 cards are spades we get 𝑃 (𝑆2) = 13/52 = 1/4.

The probability 𝑃 (𝑆2) = 1/4 may seem surprising since the value of first card
certainly affects the probabilities for the second card.

However, if we look at all possible two card sequences we will see that every card in the deck has equal probability of being the second card. Since 13 of the 52 cards are spades we get 𝑃 (𝑆2) = 13/52 = 1/4.

Another way to say this is: if we are not given value of the first card then we have to consider all possibilities for the second card.

Question 29

Draw two cards from a deck. Define the events: 𝑆1 = ‘first card is a spade’ and 𝑆2 = ‘second card is a spade’. What is the 𝑃 (𝑆1)?

Answer 29

We know that 𝑃 (𝑆1) = 1/4 because there are 52 equally probable ways to draw the first card and 13 of them are spades

Question 30

when does the law of total probability hold?

Answer 30

The law holds if we divide Ω into any number of events, so long as they are disjoint and cover all of Ω. Such a division is often called a partition of Ω.

Question 31

An urn contains 5 red balls and 2 green balls. A ball is drawn. If it’s green a red ball is added to the urn and if it’s red a green ball is added to the urn. (The original ball is not returned to the urn.) Then a second ball is drawn.

What does the tree diagram look like?

Answer 31

Question 32

What does independence mean for conditional probability?

Answer 32

𝐴 is independent of 𝐵 if 𝑃(𝐴|𝐵) = 𝑃(𝐴).

Question 33

base rate fallacy tree

Answer 33