Probability B //

Question 1

Discrete distribution

Answer 1

A r.v. X has a discrete distribution if there exists a finite or countably infinite set of values {x1, x2, … } and corresponding probabilities P1, P2, … satisfying Σ(i) Pi = 1, and
P(X = xi) = Pi for every i
P(X ∈ ℝ \ {x1, x2, … }) = 0

Question 2

Probability mass function of a discrete distribution

Answer 2

Given the set X = {x1, x2, …} where P(X = xi) = Pi > 0 for all i (the support of the distribution), the pmf is the function fx: X → (0,1], fx(xi) = Pi for every i

Question 3

Continuous distribution

Answer 3

A r.v. X has a cts distribution if there exists a function fx: ℝ → [0, ∞) with ∫(-∞, ∞) fx(x) dx = 1 such that P(a ≤ X ≤ b) = ∫(a, b) fx(x) dx for all a ≤ b (fx is the probability density function)

Question 4

Cummulative distribution function of X (discrete)

Answer 4

A function Fx: ℝ → [0,1] where Fx(u) = P(X ≤ u) = Σ(i=1, xi ≤ u) fx(xi)

fx(x) = Fx(x) - Fx(x-) for x in support,
where Fx(x-) = lim (h→0) Fx(x-h)

Question 5

Cumulative distribution function of X (cts)

Answer 5

A function Fx: ℝ → [0,1] where Fx(u) = P(X ≤ u) = ∫(-∞, u) fx(x)dx

fx(x) = (d/dx)Fx(x) where Fx diffble at x

Question 6

Rule for finding prob density of Y given Y = g(X), X cts (+ many restrictions on g)

Answer 6

If X has prob density fn fx, then

fy(y) = fx(g⁻¹(y))*(d/dy)g⁻¹(y)

Question 7

Joint distribution of X and Y

Answer 7

If X and Y are a pair of r.vs defined on a sample space (measured in the same expt), the joint dist is the probability measure on ℝ² that assigns to a subset B ⊂ ℝ² the probability P((X,Y) ∈ B)

Question 8

Discrete joint distribution and mass function

Answer 8

A joint distribution is discrete if there exists countable or finite sets {x1, x2, …} and {y1, y2, …} such that P(X ∈ {x1, x2, …} and Y ∈ {y1, y2, …}) = 1
The joint mass function is fxy(xi,yj) = P(X = xi and Y = yj)

Question 9

Cts joint distribution and density function

Answer 9

A joint distribution is continuous if there exists a joint density function fxy: ℝ² → [0,∞) so that
P(a ≤ X ≤ b and c ≤ Y ≤ d)
= ∫(a,b)∫(c,d) fxy(x,y) dydx
= ∫(c,d)∫(a,b) fxy(x,y) dxdy
(ie volume under fxy(x,y) over [a,b]x[c,d]

Question 10

Independence

Answer 10

X and Y are independent if P(X ∈ B and Y ∈ B’) = P(X ∈ B)P(Y ∈ B’) for arbitrary subsets B, B’ ∈ ℝ,
or X and Y are independent if and only if
Fxy(x,y) = Fx(x)*Fy(y)

Question 11

Convolution of discrete X and Y (dist of X + Y)

Answer 11

If X and Y are independent
P(X+Y=m) = Σ(k∈ℤ) P(X=k)P(Y=m-k)
f_x+y(m) = Σ(k∈ℤ) fx(k)fy(m-k)

Question 12

Convolution of continuous X and Y (dist of X + Y)

Answer 12

If X and Y are independent

f_x+y(z) = ∫(-∞,∞) fx(x)fy(z-x)dx = ∫(-∞,∞) fx(z-y)fy(y)dy for z ∈ ℝ

Question 13

Convolution of binomial r.vs

Answer 13

If X and Y are two independent rvs with binomial distributions, having parameters n,p and m,p respectively, then X + Y has a binomial dist with parameters m+n and p

Question 14

Expectation for discrete distribution

Answer 14

If X has a discrete dist, support {x1,x2,...} and mass fn fx(xi)=P(X=xi)
then E(X) = Σ(i)xi*fx(xi)
provided Σ|xi|fx(xi) < ∞

Question 15

Expectation for cts distribution

Answer 15

If X has a cts dist with density fx, then
E(X) = ∫(-∞,∞) x*fx(x)dx
provided ∫(-∞,∞) |x|fx(x)dx < ∞

Question 16

Expectations of functions of rvs

Answer 16

Let Y = g(X)
If X has a discrete distribution with support {x1,x2,...}
E[g(X)] = Σ(i)g(xi)fx(xi)
If X has a cts dist, 
E[g(X)] = ∫(-∞,∞) g(x)fx(x)dx

Question 17

Expectation for functions of a discrete joint dist

Answer 17

Given X,Y are a pair of rvs with respective supports {x1,x2,…} and {y1,y2,…}
E[g(x,y)] = Σ(i,j) g(xi,yj) fxy(xi,yj)

Question 18

Expectation for functions of a cts joint dist

Answer 18

E[g(X,Y)] = ∫(-∞,∞) ∫(-∞,∞) g(x,y) fxy(x,y)dxdy

Question 19

Properties of expectation (1-3)

Answer 19

Positivity
a) if P(Z ≥ 0) = 1, then E(Z) ≥ 0
b) if P(Z1 ≥ Z2) = 1, then E(Z1) ≥ E(Z2)
Expectation of constants
If P(Z=c) = 1 where c ∈ ℝ is nonrandom, then E(Z) = c
Linearity
If X1,...,Xn are rvs, a1,...,an ∈ ℝ constants,
E[Σ(i=1, n) aiXi] = Σ(i=1, n) aiE[Xi]

Question 20

Fubini property of expectation

Answer 20

If X1,…,Xn are mutually independent then

E[∏(i=1, n)Xi] = ∏(i=1, n) E[Xi]

Question 21

Cauchy-Schwartz inequality for expectation

Answer 21

If X and Y are two rvs then

|E[XY]| ≤ √(E[X²]*E[Y²]), with equality only if P(X = cY) = 1 for some c ∈ ℝ

Question 22

Markov’s inequality

Answer 22

If X is a nonnegative random variable and a > 0,

P(X ≥ a) ≤ E(X)/a

Question 23

Variance of the distribution of X

Answer 23

E[(X - μ)²] = E[X²] - (E[X])²

Question 24

(Nearly) linear property of variance

Answer 24

Var(Σ(i=1, n) ai*Xi) = Σ(i=1, n) ai²Var(Xi)

Question 25

Covariance of two rvs X and Y

Answer 25

cov(X,Y) = E[(X - μx)(Y - uy)] = E[XY] - E[X]E[Y] 
(μx = E[X], μy = E[Y])

Question 26

The variance of a convolution of X and Y

Answer 26

var(X+Y) = var(X) + var(Y) - 2cov(X,Y)

Question 27

Correlation of U and V

Answer 27

cov(U,V)/√(Var(U)*Var(V)) ∈ [-1,1]

Question 28

Expectation and variance of sums of independent variables

Answer 28

If X1, X2,…, Xn are independent, identically distributed rvs with mean μ and variance σ². If Sn = Σ(i=1, n) Xi, then
E[Sn] = nμ
Var(Sn) = σ²n

Question 29

Law of large numbers

Answer 29

Let Sn = Σ(i=1, n) Xi with X1, X2,…, Xn independent rvs having common mean μ and variance σ², then for ε > 0
P(|Sn/n - μ| ≥ ε) → 0 as n → ∞

Question 30

Central limit theorem

Answer 30

Suppose Sn = Σ(i=1, n) Xi where Xi are independent, identically distributed rvs with common mean μ and variance σ². Then for any a

Question 31

Random process

Answer 31

A collection of rvs describing the evolution of a system in time

Question 32

The memoryless property of a random variable

Answer 32

If T1, T2, … is a sequence of independent rvs, for any t ≥ 0, a ≤ b,
P(T1 ∈ [t+a,t+b] | T1 ≤ t) = P(T1 ∈ [a,b])

Question 33

Poisson process of intensity α>0

Answer 33

A collection of rvs Nt indexed by real t ≥ 0 satisfying:
1) N0 = 0
2) Nt - Ns has a poisson dist with parameter α(t-s) for any 0 ≤ s ≤ t
3) for any 0 ≤ t1 ≤ t2 ≤ … ≤ tn, the rvs Nt2-Nt1, Nt3 - Nt2, Nt4 - Nt3,… are independent
α is the mean number of occurrences per unit time

Question 34

Distribution of arrival times of a poisson process

Answer 34

If T1,T2,T3,… are the arrival times of a poisson process with intensity α, then T1, T2 - T1, T3 - T2 are independent rvs each having the exponential dist with parameter α

Question 35

Brownian motion

Answer 35

Fix μ and σ²
Suppose (Bt, t ≥ 0) is a a family of rvs with the properties
1) B0 = 0
2) Bt - Bs has the Gaussian dist with mean μ(t-s) and variance σ²(t-s)
3) if t1 ≤ t2 ≤ … ≤ tn the rvs Bt2 - Bt1, Bt3 - Bt2,… are independent

Question 36

Moment generating function φx of an rv X

Answer 36

φx(t) = E[exp(tX)] for all t for which the expectation exists
If X is discrete,
φx(t) = Σ(i=1, ∞)e^(t*xi)*fx(xi)
If X is continuous,
φx(t) = ∫(-∞,∞) e^(tx)*fx(x) dx

Question 37

Uniqueness theorem for mgfs

Answer 37

If X and Y are two random variables with mgfs φx and φy both existing in some neighbourhood (-c,c) of zero, and
φx(t) = φy(t) for all t ∈ (-ε, ε)
then X and Y have the same distribution

Question 38

Determining the mean of a distribution from its mgf

Answer 38

E[Xⁿ] = (dⁿ/dtⁿ)φx(0)

Question 39

Gaussian pdf

Answer 39

e^(-(x-μ)²/(2σ²))/(√(2π)σ)

for σ > 0, μ ∈ ℝ

Question 40

Gamma pdf

Answer 40

αⁿxⁿ⁻¹*e^(-αx)/(n-1)! if x ≥ 0
0 otherwise
for α > 0, n ∈ ℕ

Note the exponential distribution is when n = 1

Question 41

Poisson pmf

Answer 41

e^-λ*λ^k/k!, k ∈ {0,1,2,…}

for λ > 0

Question 42

Binomial pmf

Answer 42

nCkp^k(1-p)^(n-k), k ∈ {0,1,2,…n}

for p ∈ [0,1], n ∈ ℕ

Question 43

Geometric pmf

Answer 43

p(1-p)^k, k ∈ {0,1,2,3,…}

for p ∈ (0,1]

Question 44

Uniform pdf

Answer 44

(b - a)⁻¹, if a ≤ x ≤ b
0, otherwise
for a < b