Probability

Question 1

Expected value E(x)

Answer 1

Also known as the mean
Equal to the sum of all x*f(x)

Example: the expected value of the square of a fair dice roll is 1/6 + 4/6 + 9/6 + 16/6 + 25/6 + 36/6 = 91/6

Question 2

Variance V(X)

Answer 2

The measure of dispersion or variability.
V(X) = E[(X-μ)^2] = (sum of all f(x)*x^2) - μ^2

Question 3

Formula linking variance and expected value

Answer 3

V(X) = E(X^2) - E(X)^2

Question 4

Expected value of multiple variables

Answer 4

E(X + Y + Z) = E(X) + E(Y) + E(Z)

Question 5

Variance of multiple variables

Answer 5

V(X + Y + Z) = V(X) + V(Y) + V(Z)
independent variables only

Question 6

E(h(X))

Answer 6

sum of all h(x)f(x)
So E(x^2) = sum of all x^2f(x)

Question 7

E(aX + b)

Answer 7

a*E(X) + b

Question 8

V(aX + b)

Answer 8

V(X)*a^2

Question 9

P(A or B)

Answer 9

P(A) + P(B) - P(A and B)

Question 10

P(A given B)

Answer 10

P(A and B)/P(B)

Question 11

Number of ways of choosing r elements from a set of n, and taking order into account

Answer 11

P(n, r) = n!/(n-r)!

Question 12

Our set has n objects, with n1 of one type and n2 of a second type, …, nk of a kth type.
Number of possible permutations?

Answer 12

n!/(n1! * n2! * … * nk!)

Question 13

Number of combinations (unordered arrangements) of r elements in a set of n

Answer 13

C(n, r) = n!/(r! * (n-r)!)

Question 14

Two events are independent…

Answer 14

if and only if P(A and B) = P(A)P(B)
==> P(A given B) = P(B given A) = P(A)P(B)

Question 15

Bayes’ theorem

Answer 15

P(A given B) = P(B given A) * P(A)/P(B)

Question 16

General version of Bayes’ theorem for mutually exclusive events

Answer 16

P(Ek|A) = P(Ek)P(A|Ek) / [P(E1)P(A|E1) + … + P(En)*P(A|En)]

Question 17

Bernoulli distribution

Answer 17

This is when there’s one random experiment with two possible outcomes
Notation: X~Ber(p), where p=P(X=1)
E(X) = p
V(X) = p(1-p)

Question 18

Discrete binomial distribution

Answer 18

X is the number of successful trials out of n, each with probability p
Notation: X~Bin(n, p)
P(X=k) = C(n, k) * p^k * (1-p)^(n-k)
E(x) = np
V(X) = np(1-p)

Question 19

Negative binomial distribution

Answer 19

X is the number of trials until r successes occur
Notation: X~NegBin(r, p)
P(X=k) = C(k-1, r-1) * p^r * (1-p)^(k-r)
E(X) = r/p
V(X) = r*(1-p)/p^2

Question 20

Discrete geometric distribution

Answer 20

X is the number of Bernoulli trials until one success
Notation: X~Geom(p)
P(X=k) = p(1-p)^(k-1)
E(X) = 1/p
V(X) = (1-p)/p^2

Question 21

Standard deviation definition

Answer 21

Measure of the amount of variation of a random variable expected about its mean.
Equal to the square root of variance

Question 22

Poisson distribution

Answer 22

X is the number of times an event is likely to happen within a specific period of time. λ is the number of times the event is usually expected to happen during that period of time.
Notation: X~Poisson(λ)
P(X=k) = (e^-λ * λ^k)/k!
E(X) = V(X) = λ

Question 23

Using the Poisson distribution to model the binomial distribution

Answer 23

𝜆 = np
when n > 20
and np < 5 or n(1-p) < 5

Question 24

Mean of a continuous random variable

Answer 24

E(X) = integral from -inf to inf of x*f(x)

Question 25

Variance of a continuous random variable

Answer 25

s^2 = V(X) = E[(X-m)^2]
int from -inf to inf of f(x)*(x-m)^2

Question 26

Exponential distribution

Answer 26

It is used to model the time elapsing between events, and the probability of occurrence of an event after a period of time. L is the average number of events occurring in a time unit.
Notation: X~Exp(L)
P(X <= k) = 1 - e^-Lx (0 if x < 0)
E(X) = 1/L
V(X) = 1/L^2

Question 26

Discrete geometric distribution

Answer 26

All samples in the range have the same probability, meaning
E(X) = (b+a)/2
V(X) = (b-a)^2/12

Question 27

Normal distribution

Answer 27

Standard bell curve
Notation: X~N(μ, σ^2)
E(X) = μ
V(X) = σ^2

The area between μ-σ and μ+σ is about 68%, the area between μ-2σ and μ+2σ is about 95%, and the area between μ-3σ and μ+3σ is about 99.7%.

Question 28

Standard normal distribution

Answer 28

Special case of the normal distribution with μ = 0 and σ^2 = 1. P(Z <= z) is written as Φ(z)

We convert a normal distribution X to standard ND by setting Z = (X-μ)/σ. We call this the Z-score, a measure of how many standard deviations X is from the mean.

Question 29

Using normal distribution to approximate the binomial distribution

Answer 29

When n is large and p is around 0.5.

X~Bin(n, p) => P(X <= x) = Φ((x-np)/sqrt(npq))

Question 30

Using normal distribution to approximate the Poisson distribution

Answer 30

X~Poisson(L) => P(X <= x) = Φ((x-L)/sqrt(L))
Good for L > 5

Question 31

Joint probability

Answer 31

P(X=x, Y=y) = P(X=x and Y=y) = joint probability of X=x and Y=y

f(x, y) = P(X=x, Y=y)
F(x, y) = P(X <= x, Y <= y)

The JPMF has the following properties: always positive, sums to 1

Question 32

Marginal probability distribution

Answer 32

Removing one of the variables from the JPMF
P(X=k) = sum of all the probability pairs (X, Y) where X=k

P(Y=y | X=x) = P(X=x, Y=y)/P(X=x)

Question 33

JPDF of the continuous random variable fXY(x, y)

Answer 33

P(a <= X <= b, c <= Y <= d) =
int a to b (int c to d fXY(x, y) dy) dx)

Question 34

Marginal probability distribution: continuous edition

Answer 34

fX(x) = int fXY(x, y)dy
fY(y) = int fXY(x, y)dx

Question 35

When are X and Y independent?

Answer 35

One of the following statements holds:
fX(x)fY(y) = fXY(x,y)
f(Y=y | X=X) = fY(y) when fX(x)>0
or the other way round
P(X in A given Y in B) = P(X in A)P(Y in B) for any two sets A and B

Question 36

What is covariance?

Answer 36

A measure of the strength of the linear relationship between two variables
cov(X, Y) = σXY = E[XY] - E[X]E[Y]
cov(X, Y) = E[(X-E(X))(Y-E(Y))]

Question 37

Covariance rules

Answer 37

Cov(aX+b, cY+d) = acCov(X, Y)
Cov(X1+X2, Y) = Cov(X1, Y) + Cov(X2, Y)
Cov(X, X) = V(X)
If X and Y are independent, then Cov(X, Y) = 0
It’s always between -1 and 1

Question 38

Correlation coefficient

Answer 38

Corr(X, Y) = pXY = cov(X, Y)/sqrt(V(X)*V(Y))

Question 39

Markov’s inequality

Answer 39

P(X >= a) <= E(X)/a
a > 0

Question 40

Chebyshev’s inequality

Answer 40

P(|X- 𝜇| >= k) <= 𝜎^2 / k^2, for all k>0

Question 41

Variance of sample

Answer 41

𝜎^2/n
population variance/sample size

Question 42

Central limit theorem

Answer 42

Consider n random variables. Suppose they all have distribution F with E(Xi) = 𝜇 and V(Xi) = 𝜎^2, and let X* be the average of all the Xn’s. As n approaches infinity, we will have X~N(𝜇, 𝜎^2 / n). This means that the distribution of the sample mean will be approximately a normal distribution.

Question 43

Bias and variance of an estimator

Answer 43

b(t) = E(t) – t
V(t) = E[(t - E(t*))^2]

Question 44

Confidence intervals

Answer 44

Depending on the confidence value we want to have, the value of z changes

P(x* - |z|𝜎/sqrt(n) < x < x* + |z|𝜎/sqrt(n)) = 1-a

z = the Z-score when the area to the left is 1 - a/2
(a is small)

Default is two-sided, if one-sided then t <= t* + |z|𝜎/sqrt(n)

Question 45

Hypothesis testing steps

Answer 45

Calculate the probability that the sample statistic of a random sample is <= t* (can also be >=), assuming that H0 is true. Call this the p-value.
If the p-value is smaller than a significance level, then reject H0 and accept H1.

Question 46

Test statistic

Answer 46

Value used in making a decision about the null hypothesis