basic probability

Question 1

probability

Answer 1

logically self contained
few rules, answers follow from rules
computations can be tricky

Question 2

statistics

Answer 2

apply probability to draw conclusions from data
can be messy
involves art and science

Question 3

probability example

Answer 3

you have a fair coin. you will toss it 100 times. What is the probability of 60 or more heads?

• random process fully known ( p(heads) = 0.5)
• outcome is unknown

Question 4

statistics example

Answer 4

you have a coin of unknown provenance. to investigate its fairness, you toss it 100 times. Suppose you count 60 heads.

• multiple ways to do this
• different statisticians may draw different conclusions
• outcome is known (60 heads)
• need to illuminate random process

Question 5

what are the schools of statistics

Answer 5

frequentists and bayesian

Question 6

frequentists

Answer 6

probability measures frequency of various outcomes of an experiment
fair coin has 50% chance of heads means if we toss it many times we expect to see that percentage of heads

Question 7

bayesians

Answer 7

probability is an abstract concept measuring the state of knowledge or degree of belief in given proposition
consider a range of values each with its own probability of being true

Question 8

principle of equally likely outcomes

Answer 8

suppose there are n possible outcomes for an experiment and each is equally probable

if there are k desirable outcomes, then the probability of a desirable outcome is k/n

Question 9

set S

Answer 9

collection of elements

Question 10

element

Answer 10

𝑥 ∈ S to mean element x is in set S

Question 11

subset

Answer 11

𝐴 ⊂ 𝑆

𝐴 is a subset of 𝑆 if all of its elements are in S

Question 12

complement

Answer 12

The complement of 𝐴 in 𝑆 is the set of elements of 𝑆 that are not in 𝐴.

We write this as
𝐴𝑐 or 𝑆 − 𝐴.

Question 13

union

Answer 13

The union of 𝐴 and 𝐵 is the set of all elements in 𝐴 or 𝐵 (or both). We write this as 𝐴 ∪ 𝐵.

Question 14

intersection

Answer 14

The intersection of 𝐴 and 𝐵 is the set of all elements in both 𝐴 and 𝐵. We
write this as 𝐴 ∩ 𝐵.

Question 15

empty set

Answer 15

The empty set is the set with no elements. We denote it ∅.

Question 16

disjoint

Answer 16

𝐴 and 𝐵 are disjoint if they have no common elements. That is, if 𝐴 ∩ 𝐵 = ∅

Question 17

difference

Answer 17

The difference of 𝐴 and 𝐵 is the set of elements in 𝐴 that are not in 𝐵. We
write this as 𝐴 − 𝐵.

Question 18

deMorgan’s laws

Answer 18

(𝐴 ∪ 𝐵)𝑐 = 𝐴𝑐 ∩ 𝐵𝑐
(𝐴 ∩ 𝐵)𝑐 = 𝐴𝑐 ∪ 𝐵𝑐

the first law says everything not in (𝐴 or 𝐵) is the same set as everything that’s (not in 𝐴) and (not in 𝐵).

The second law is similar.

Question 19

venn diagrams

Answer 19

visualize set operations

Question 20

s

Answer 20

Question 21

L

Answer 21

Question 22

R

Answer 22

Question 23

𝐿 ∪ 𝑅

Answer 23

Question 24

𝐿 ∩ 𝑅

Answer 24

Question 25

L^C

Answer 25

Question 26

𝐿 − 𝑅

Answer 26

Question 27

Proof of DeMorgan’s Laws

Answer 27

Question 28

products of sets

Answer 28

𝑆 × 𝑇 = {(𝑠,𝑡)| 𝑠 ∈ 𝑆,𝑡 ∈ 𝑇 }.

the set of ordered pairs (𝑠,𝑡) such that 𝑠 is in 𝑆 and 𝑡
is in T

Question 29

example of set product

Answer 29

Question 30

example of set product 2

Answer 30

also illustrates that 𝐴 ⊂ 𝑆 and 𝐵 ⊂ 𝑇 then 𝐴 × 𝐵 ⊂ 𝑆 × 𝑇 .

Question 31

counting

Answer 31

If 𝑆 is finite, we use |𝑆| or #𝑆 to denote the number of elements of 𝑆.

Two useful counting principles:
1. inclusion-exclusion principle
2. rule of product

Question 32

inclusion-exclusion principle

Answer 32

|𝐴 ∪ 𝐵| = |𝐴| + |𝐵| − |𝐴 ∩ 𝐵|

|𝐴| is the number of dots in 𝐴 and likewise for the other sets.

The figure shows that |𝐴|+|𝐵|
double-counts |𝐴 ∩ 𝐵|, which is why |𝐴 ∩ 𝐵| is subtracted off

Question 33

In a band of singers and guitarists, seven people sing, four play the guitar,
and two do both. How big is the band?

Answer 33

: Let 𝑆 be the set singers and 𝐺 be the set guitar players. The inclusion-exclusion
principle says
size of band = |𝑆 ∪ 𝐺| = |𝑆| + |𝐺| − |𝑆 ∩ 𝐺| = 7 + 4 − 2 = 9.

Question 34

rule of product

Answer 34

𝑛 ways to perform action 1
𝑚 ways to perform action
2,
then there are 𝑛 ⋅ 𝑚 ways to perform action 1 followed by action 2.

Question 35

If you have 3 shirts and 4 pants then how many outfits can you make?

Answer 35

multiplication rule/ rule of product

3 * 4 = 12

Question 36

There are 5 competitors in the 100m final at the Olympics. In how many ways can the gold, silver, and bronze medals be awarded?

Answer 36

There are 5 ways to award the gold.

Once that is awarded there are 4 ways to
award the silver and then 3 ways to award the bronze:

5 ⋅ 4 ⋅ 3 = 60 ways.

Note that the choice of gold medalist affects who can win the silver, but the number of
possible silver medalists is always four.

Question 37

permutation

Answer 37

A permutation of a set is a particular ordering of its elements. For example, the set {𝑎, 𝑏, 𝑐} has six permutations: 𝑎𝑏𝑐, 𝑎𝑐𝑏, 𝑏𝑎𝑐, 𝑏𝑐𝑎, 𝑐𝑎𝑏, 𝑐𝑏𝑎.

find by listing, or find buy using rule of product

Note that 𝑎𝑏𝑐 and 𝑎𝑐𝑏 count as distinct permutations. That is, for permutations the order matters.

permutations are lists

Question 38

what is the number of permutations of a set of k elements?

Answer 38

𝑘! = 𝑘 ⋅ (𝑘 − 1) ⋯ 3 ⋅ 2 ⋅ 1

Question 39

combinations

Answer 39

order does not matter

combinations are sets.

Question 40

List all the combinations of 3 elements out of the set {𝑎, 𝑏, 𝑐, 𝑑}.

Answer 40

Such a combination is a collection of 3 elements without regard to order. So, 𝑎𝑏𝑐 and 𝑐𝑎𝑏 both represent the same combination.

We can list all the combinations by listing all the subsets of exactly 3 elements.

{𝑎, 𝑏, 𝑐} {𝑎, 𝑏, 𝑑} {𝑎, 𝑐, 𝑑} {𝑏, 𝑐, 𝑑}

There are only 4 combinations.

Contrast this with the 24 permutations in the list example.

The factor of 6 comes because every combination of 3 things can be written in 6 different orders

Question 41

𝑛𝑃𝑘

Answer 41

number of permutations (lists) of 𝑘 distinct elements from a set of size n

n!/(n-k)!

Question 42

nCk

Answer 42

n!/(k!*(n-k)!) = nPk/k!

n choose K

Question 43

experiment

Answer 43

repeatable procedure with well defined possible outcomes

Question 44

sample space

Answer 44

the set of all possible outcomes.

We usually denote the sample space by Ω, sometimes by 𝑆.

Question 45

event

Answer 45

a subset of the sample space

The probability of an event 𝐸 is computed by adding up the probabilities of all of the outcomes in E

Question 46

probability function

Answer 46

a function giving the probability for each outcome.

Question 47

probability density

Answer 47

continuous distribution of probabilities

Question 48

random variable

Answer 48

random numerical outcome

Question 49

discrete sample space

Answer 49

is one that is listable, it can be either finite or infinite

{H, T},
{1, 2, 3},
{1, 2, 3, 4, …},
{2, 3, 5, 7, 11, 13, 17…}

Question 50

probability function

Answer 50

For a discrete sample space 𝑆 a probability function 𝑃 assigns to each outcome 𝜔 a number
𝑃(𝜔) called the probability of 𝜔. 𝑃 must satisfy two rules:

• 0 ≤ 𝑃 (𝜔) ≤ 1 (probabilities are between 0 and 1).
• The sum of the probabilities of all possible outcomes is 1 (something must occur)

Question 51

probability rule 1

Answer 51

𝑃(𝐴𝑐) = 1 − 𝑃(𝐴)

𝐴 and 𝐴𝑐 split Ω into two non-overlapping regions. Since the total probability
𝑃(Ω) = 1 this rule says that the probability of 𝐴 and the probability of ’not 𝐴’ are complementary, i.e. sum to 1.

Question 52

probability rule 2

Answer 52

If 𝐿 and 𝑅 are disjoint then 𝑃 (𝐿 ∪ 𝑅) = 𝑃 (𝐿) + 𝑃 (𝑅).

𝐿 and 𝑅 split 𝐿 ∪ 𝑅 into two non-overlapping regions. So the probability of 𝐿 ∪ 𝑅 is is split between 𝑃(𝐿) and 𝑃(𝑅)

Question 53

Rule 3

Answer 53

If 𝐿 and 𝑅 are not disjoint, we have the inclusion-exclusion principle

𝑃 (𝐿 ∪ 𝑅) = 𝑃 (𝐿) + 𝑃 (𝑅) − 𝑃 (𝐿 ∩ 𝑅)

In the sum 𝑃(𝐿) + 𝑃(𝑅) the overlap 𝑃 (𝐿 ∩ 𝑅) gets counted twice. So 𝑃(𝐿) +
𝑃(𝑅) − 𝑃 (𝐿 ∩ 𝑅) counts everything in the union exactly once.

Question 55

Countable (notations..) and Uncountable set.

Answer 55

A set that is either finite or has the same cardinality as the set of positive integers is called
countable. A set that is not countable is called uncountable. When an infinite set S is countable, we denote the cardinality of S by ℵ0 (where ℵ is aleph, the first letter of the Hebrew
alphabet).
We write |S| = ℵ0 and say that S has cardinality “aleph null.”

Question 56

P (empty set) =

Answer 56

0

Question 57

Probability

Answer 57

A probability is a function P that assigns to each event E in the sample space S a number P(E) called the probability of event E.