intermediate probability Flashcards
conditional probability
how does the probability of the event changes if we have extra information?
The conditional probability of π΄ knowing that π΅ occurred is written
π(π΄|π΅)
the conditional probability of π΄ given π΅
formal definition of conditional probability
multiplication rule
π(π΄ β© π΅) = π(π΄|π΅) β π(π΅).
This is simply a rewriting of the definition in Equation (1) of conditional probability.
We will see that our use of the multiplication rule is very similar to our use of the rule of
product in counting.
law of total probability
Suppose the sample space Ξ© is divided into 3 disjoint events π΅1, π΅2, π΅3 (see the figure
below). Then for any event A:
π (π΄) = π (π΄ β© π΅1) + π (π΄ β© π΅2) + π (π΄ β© π΅3)
π(π΄) = π(π΄|π΅1) π(π΅1) + π(π΄|π΅2) π(π΅2) + π(π΄|π΅3) π(π΅3)
An urn contains 5 red balls and 2 green balls. Two balls are drawn one after
the other. What is the probability that the second ball is red?
The sample space is Ξ© = {rr, rg, gr, gg}.
Let π
1 be the event βthe first ball is redβ, πΊ1 = βfirst ball is greenβ, π
2 = βsecond ball is
redβ, πΊ2 = βsecond ball is greenβ. We are asked to find π(π
2).
Letβs compute this same value using the law of total probability (3). First, weβll find the conditional probabilities. This is a simple counting exercise.
π(π
2|π
1) = 4/6, π(π
2|πΊ1) = 5/6.
Since π
1 and πΊ1 partition Ξ© the law of total probability says
π(π
2) = π(π
2|π
1)π(π
1) + π(π
2|πΊ1)π(πΊ1) = 5/7
probability urns
In probability and statistics, an urn problem is an idealized mental exercise
in which some objects of real interest (such as atoms, people, cars, etc.) are
represented as colored balls in an urn or other container. One pretends to draw (remove) one or more balls from the urn; the goal is to determine the probability of drawing one color or another, or some other properties. A key parameter is whether each ball is returned to the urn after each draw.
An urn contains 5 red balls and 2 green balls. A ball is drawn. If itβs green
a red ball is added to the urn and if itβs red a green ball is added to the urn. (The original ball is not returned to the urn.) Then a second ball is drawn. What is the probability the second ball is red?
The law of total probability says that π(π
2) can be computed using the expression in Equation (4). Only the values for the probabilities will change. We have
π(π
2|π
1) = 4/7, π(π
2|πΊ1) = 6/7.
Therefore, π(π 2) = π(π 2|π 1)π(π 1) + π(π 2|πΊ1)π(πΊ1) = (4/7)*(5/7) +(6/7)(2/7) = 32/49
trees to organize computation
consider the case of the urn with 5 red balls and two green balls. if the first ball drawn is green, a red ball is added to the urn, and if the first ball drawn is red, a green ball is added to the urn. what is the probability of drawing a red ball second?
probability tree
node - Each dot is called a node.
levels - The tree is organized by levels.
The top node (root node) is at level 0. The next layer down is level 1 and so on. Each level shows the outcomes at one stage of the game. Level 1 shows the possible outcomes of the first draw. Level 2 shows the possible outcomes of the second draw starting from each node in level 1. Probabilities are written along the branches. The probability of π 1 (red on the first draw) is 5/7.
It is written along the branch from the root node to the one labeled π
1. At the
next level we put in conditional probabilities. The probability along the branch from π
1 to π
2 is π(π
2|π
1) = 4/7. It represents the probability of going to node π
2 given that you are already at π
1.
The multiplication rule says that the probability of getting to any node is just the product of the probabilities along the path to get there. For example, the node labeled π
2 at the far left really represents the event π
1 β© π
2 because it comes from the π
1 node. The multiplication rule now says
π(π 1 β© π 2) = π (π 1) β π (π 2|π 1)
The law of total probability is just the statement that π(π 2) is the sum of the probabilities of all paths leading to π 2 (the two circled nodes in the figure).
independence
Two events are independent if knowledge that one occurred does not change the probability that the other occurred.
Informally, events are independent if they do not influence one another
formulation of independence
π(π΄ β© π΅) = π (π΄) β π (π΅)
- If π (π΅) β 0 then π΄ and π΅ are independent if and only if π(π΄|π΅) = π(π΄).
- If π (π΄) β 0 then π΄ and π΅ are independent if and only if π(π΅|π΄) = π(π΅).
Toss a fair coin twice. Let π»1 = βheads on first tossβ and let π»2 = βheads on
second tossβ. Are π»1 and π»2 independent?
Since π»1 β© π»2 is the event βboth tosses are headsβ we have
π(π»1 β© π»2) = 1/4 = π(π»1)π(π»2).
Therefore the events are independent.
Toss a fair coin 3 times. Let π»1 = βheads on first tossβ and π΄ = βtwo heads
totalβ. Are π»1 and π΄ independent?
We know that π(π΄) = 3/8. Since this is not 0 we can check if the formula π(π΄|π΅) = π(π΄) holds.
Now, π»1 = {HHH, HHT, HTH, HTT} contains exactly two outcomes (π»π»π , π»π π») from π΄, so we have π(π΄|π»1) = 2/4.
Since π(π΄|π»1) β π(π΄) these events are not independent.
Draw one card from a standard deck of playing cards. Letβs examine the
independence of 3 events βthe card is an aceβ, βthe card is a heartβ and βthe card is redβ.
Define the events as π΄ = βaceβ, π» = βheartsβ, π = βredβ.
(a) We know that π(π΄) = 4/52 (4 out of 52 cards are aces), π(π΄|π») = 1/13 (1 out of 13 hearts are aces). Since π (π΄) = π (π΄|π») we have that π΄ is independent of π».
(b) π(π΄|π ) = 2/26 = 1/13 = π(π΄). So π΄ is independent of π . That is, whether the card is an ace is independent of whether it is red.
(c) Finally, what about π» and π ? Since π (π») = 1/4 and π (π»|π ) = 1/2, π» and π are not independent. We could also see this the other way around: π(π ) = 1/2 and π (π |π») = 1, so π» and π are not independent. That is, the suit of a card is not independent of the color of the cardβs suit.
paradoxes of independence
An event π΄ with probability 0 is independent of itself, since in this case both sides of equation (6) are 0. This appears paradoxical because knowledge that π΄ occurred certainly gives information about whether π΄ occurred. We resolve the paradox by noting that since π(π΄) = 0 the statement βπ΄ occurredβ is vacuous.
Think: For what other value(s) of π(π΄) is π΄ independent of itself?
bayeβs theorem
For two events π΄ and π΅ Bayesβ theorem (also called Bayesβ rule and Bayesβ
formula) say (reference the fig)
- Bayesβ rule tells us how to βinvertβ conditional probabilities, i.e. to find
π(π΅|π΄) from π(π΄|π΅). - In practice, π(π΄) is often computed using the law of total probability.
proof of bayeβs theorem
The key point is that π΄ β© π΅ is symmetric in π΄ and π΅. So the multiplication rule says
π(π΅|π΄) β
π(π΄) = π(π΄ β© π΅) = π(π΄|π΅) β
π(π΅).
Now divide through by π(π΄) to get Bayesβ rule.
A common mistake is to confuse the meanings of π(π΄|π΅) and π(π΅|π΄). They can be very
different. This is illustrated in the next example.
Toss a coin 5 times. Let π»1 = βfirst toss is headsβ and let π»π΄ = βall 5 tosses
are headsβ. Then π(π»1|π»π΄) = 1 but π(π»π΄|π»1) = 1/16.
use Bayesβ theorem to compute π(π»1|π»π΄) using π(π»π΄|π»1)
The terms are π(π»π΄|π»1) = 1/16, π(π»1) = 1/2, π(π»π΄) = 1/32.
base rate fallacy
The base rate fallacy is one of many examples showing that itβs easy to confuse the meaning of π(π΅|π΄) and π(π΄|π΅) when a situation is described in words. This is one of the key
examples from probability and it will inform much of our practice and interpretation of statistics. You should strive to understand it thoroughly.
Consider a routine screening test for a disease. Suppose the frequency of the disease in the population (base rate) is 0.5%. The test is fairly accurate with a 5% false positive rate and a 10% false negative rate.
You take the test and it comes back positive. What is the probability that you have the disease?
π·+ = βyou have the diseaseβ
π·β = βyou do not have the disease
π + = βyou tested positiveβ
π β = βyou tested negativeβ.
We are given π(π·+) = 0.005 and therefore π(π·β) = 0.995. The false positive and false
negative rates are (by definition) conditional probabilities.
π(false positive) = π(π +|π·β) = 0.05 and π(false negative) = π(π β|π·+) = 0.1.
The complementary probabilities are known as the true negative and true positive rates:
π(π β|π·β) = 1 β π(π +|π·β) = 0.95
π(π +|π·+) = 1 β π(π β|π·+) = 0.9.
Using bayeβs theorem on these probabilities to get π(π·+|π +) we calculate around 8.3%!!
remarks on base rate fallacy
This is called the base rate fallacy because the base rate of the disease in the population is so low that the vast majority of the people taking the test are healthy, and even with an accurate test most of the positives will be healthy people. Ask your doctor
for his/her guess at the odds.
To summarize the base rate fallacy with specific numbers 95% of all tests are accurate does not imply 95% of positive tests are accurate
We will refer back to this example frequently. It and similar examples are at the heart of many statistical misunderstandings.
toss a fair coin 3 times
what is the probability of 3 heads?
Sample space Ξ© = {π»π»π», π»π»π , π»ππ», π»ππ , ππ»π», ππ»π , πππ», πππ }.
All outcomes are equally probable, so π (3 heads) = 1/8.
toss a fair coin 3 times
Suppose the first toss was heads
given this info how should we compute the probability of 3 heads?
conditional probability
takes into account additional conditions
the conditional probability of A knowing that B occurred is written
π (π΄|π΅)
βthe conditional probability of π΄ given π΅β
Draw two cards from a deck. Define the events: π1 = βfirst card is a spadeβ and π2 = βsecond card is a spadeβ. What is the π (π2|π1)?
Refer to other cards where we compute π (π1), π (π2) and π (π1 β© π2). We use the formula above, and get
Draw two cards from a deck. Define the events: π1 = βfirst card is a spadeβ and π2 = βsecond card is a spadeβ.
What is π(π1β©π2)?
We compute π (π1 β© π2) by counting:
Number of ways to draw a spade followed by a second spade: 13 β 12.
Number of ways to draw any card followed by any other card: 52 β 51.
Draw two cards from a deck. Define the events: π1 = βfirst card is a spadeβ and π2 = βsecond card is a spadeβ. What is the π (π2)?
Since 13 of the 52 cards are spades we get π (π2) = 13/52 = 1/4.
The probability π (π2) = 1/4 may seem surprising since the value of first card
certainly affects the probabilities for the second card.
However, if we look at all possible two card sequences we will see that every card in the deck has equal probability of being the second card. Since 13 of the 52 cards are spades we get π (π2) = 13/52 = 1/4.
Another way to say this is: if we are not given value of the first card then we have to consider all possibilities for the second card.
Draw two cards from a deck. Define the events: π1 = βfirst card is a spadeβ and π2 = βsecond card is a spadeβ. What is the π (π1)?
We know that π (π1) = 1/4 because there are 52 equally probable ways to draw the first card and 13 of them are spades
when does the law of total probability hold?
The law holds if we divide Ξ© into any number of events, so long as they are disjoint and cover all of Ξ©. Such a division is often called a partition of Ξ©.
An urn contains 5 red balls and 2 green balls. A ball is drawn. If itβs green a red ball is added to the urn and if itβs red a green ball is added to the urn. (The original ball is not returned to the urn.) Then a second ball is drawn.
What does the tree diagram look like?
What does independence mean for conditional probability?
π΄ is independent of π΅ if π(π΄|π΅) = π(π΄).
base rate fallacy tree
