basic probability Flashcards
probability
logically self contained
few rules, answers follow from rules
computations can be tricky
statistics
apply probability to draw conclusions from data
can be messy
involves art and science
probability example
you have a fair coin. you will toss it 100 times. What is the probability of 60 or more heads?
- random process fully known ( p(heads) = 0.5)
- outcome is unknown
statistics example
you have a coin of unknown provenance. to investigate its fairness, you toss it 100 times. Suppose you count 60 heads.
- multiple ways to do this
- different statisticians may draw different conclusions
- outcome is known (60 heads)
- need to illuminate random process
what are the schools of statistics
frequentists and bayesian
frequentists
probability measures frequency of various outcomes of an experiment
fair coin has 50% chance of heads means if we toss it many times we expect to see that percentage of heads
bayesians
probability is an abstract concept measuring the state of knowledge or degree of belief in given proposition
consider a range of values each with its own probability of being true
principle of equally likely outcomes
suppose there are n possible outcomes for an experiment and each is equally probable
if there are k desirable outcomes, then the probability of a desirable outcome is k/n
set S
collection of elements
element
π₯ β S to mean element x is in set S
subset
π΄ β π
π΄ is a subset of π if all of its elements are in S
complement
The complement of π΄ in π is the set of elements of π that are not in π΄.
We write this as
π΄π or π β π΄.
union
The union of π΄ and π΅ is the set of all elements in π΄ or π΅ (or both). We write this as π΄ βͺ π΅.
intersection
The intersection of π΄ and π΅ is the set of all elements in both π΄ and π΅. We
write this as π΄ β© π΅.
empty set
The empty set is the set with no elements. We denote it β .
disjoint
π΄ and π΅ are disjoint if they have no common elements. That is, if π΄ β© π΅ = β
difference
The difference of π΄ and π΅ is the set of elements in π΄ that are not in π΅. We
write this as π΄ β π΅.
deMorganβs laws
(π΄ βͺ π΅)π = π΄π β© π΅π
(π΄ β© π΅)π = π΄π βͺ π΅π
the first law says everything not in (π΄ or π΅) is the same set as everything thatβs (not in π΄) and (not in π΅).
The second law is similar.
venn diagrams
visualize set operations
s
L
R
πΏ βͺ π
πΏ β© π
L^C
πΏ β π
Proof of DeMorganβs Laws
products of sets
π Γ π = {(π ,π‘)| π β π,π‘ β π }.
the set of ordered pairs (π ,π‘) such that π is in π and π‘
is in T
example of set product
example of set product 2
also illustrates that π΄ β π and π΅ β π then π΄ Γ π΅ β π Γ π .
counting
If π is finite, we use |π| or #π to denote the number of elements of π.
Two useful counting principles:
1. inclusion-exclusion principle
2. rule of product
inclusion-exclusion principle
|π΄ βͺ π΅| = |π΄| + |π΅| β |π΄ β© π΅|
|π΄| is the number of dots in π΄ and likewise for the other sets.
The figure shows that |π΄|+|π΅|
double-counts |π΄ β© π΅|, which is why |π΄ β© π΅| is subtracted off
In a band of singers and guitarists, seven people sing, four play the guitar,
and two do both. How big is the band?
: Let π be the set singers and πΊ be the set guitar players. The inclusion-exclusion
principle says
size of band = |π βͺ πΊ| = |π| + |πΊ| β |π β© πΊ| = 7 + 4 β 2 = 9.
rule of product
π ways to perform action 1
π ways to perform action
2,
then there are π β
π ways to perform action 1 followed by action 2.
If you have 3 shirts and 4 pants then how many outfits can you make?
multiplication rule/ rule of product
3 * 4 = 12
There are 5 competitors in the 100m final at the Olympics. In how many ways can the gold, silver, and bronze medals be awarded?
There are 5 ways to award the gold.
Once that is awarded there are 4 ways to
award the silver and then 3 ways to award the bronze:
5 β 4 β 3 = 60 ways.
Note that the choice of gold medalist affects who can win the silver, but the number of
possible silver medalists is always four.
permutation
A permutation of a set is a particular ordering of its elements. For example, the set {π, π, π} has six permutations: πππ, πππ, πππ, πππ, πππ, πππ.
find by listing, or find buy using rule of product
Note that πππ and πππ count as distinct permutations. That is, for permutations the order matters.
permutations are lists
what is the number of permutations of a set of k elements?
π! = π β (π β 1) β― 3 β 2 β 1
combinations
order does not matter
combinations are sets.
List all the combinations of 3 elements out of the set {π, π, π, π}.
Such a combination is a collection of 3 elements without regard to order. So, πππ and πππ both represent the same combination.
We can list all the combinations by listing all the subsets of exactly 3 elements.
{π, π, π} {π, π, π} {π, π, π} {π, π, π}
There are only 4 combinations.
Contrast this with the 24 permutations in the list example.
The factor of 6 comes because every combination of 3 things can be written in 6 different orders
πππ
number of permutations (lists) of π distinct elements from a set of size n
n!/(n-k)!
nCk
n!/(k!*(n-k)!) = nPk/k!
n choose K
experiment
repeatable procedure with well defined possible outcomes
sample space
the set of all possible outcomes.
We usually denote the sample space by Ξ©, sometimes by π.
event
a subset of the sample space
The probability of an event πΈ is computed by adding up the probabilities of all of the outcomes in E
probability function
a function giving the probability for each outcome.
probability density
continuous distribution of probabilities
random variable
random numerical outcome
discrete sample space
is one that is listable, it can be either finite or infinite
{H, T},
{1, 2, 3},
{1, 2, 3, 4, β¦},
{2, 3, 5, 7, 11, 13, 17β¦}
probability function
For a discrete sample space π a probability function π assigns to each outcome π a number
π(π) called the probability of π. π must satisfy two rules:
- 0 β€ π (π) β€ 1 (probabilities are between 0 and 1).
- The sum of the probabilities of all possible outcomes is 1 (something must occur)
probability rule 1
π(π΄π) = 1 β π(π΄)
π΄ and π΄π split Ξ© into two non-overlapping regions. Since the total probability
π(Ξ©) = 1 this rule says that the probability of π΄ and the probability of βnot π΄β are complementary, i.e. sum to 1.
probability rule 2
If πΏ and π are disjoint then π (πΏ βͺ π ) = π (πΏ) + π (π ).
πΏ and π split πΏ βͺ π into two non-overlapping regions. So the probability of πΏ βͺ π is is split between π(πΏ) and π(π )
Rule 3
If πΏ and π are not disjoint, we have the inclusion-exclusion principle
π (πΏ βͺ π ) = π (πΏ) + π (π ) β π (πΏ β© π )
In the sum π(πΏ) + π(π
) the overlap π (πΏ β© π
) gets counted twice. So π(πΏ) +
π(π
) β π (πΏ β© π
) counts everything in the union exactly once.
Countable (notations..) and Uncountable set.
A set that is either finite or has the same cardinality as the set of positive integers is called
countable. A set that is not countable is called uncountable. When an infinite set S is countable, we denote the cardinality of S by β΅0 (where β΅ is aleph, the first letter of the Hebrew
alphabet).
We write |S| = β΅0 and say that S has cardinality βaleph null.β
P (empty set) =
0
Probability
A probability is a function P that assigns to each event E in the sample space S a number P(E) called the probability of event E.
