Foundations of probability and statistics Flashcards
Properties of Event spaces
- contain the empty set
- To be closed under complementation
- Closed under finite unions
De morgans law
(A∩B)^c =A^c ∪ B^c
Probability measure
A probability measure is a function which assigns a numerical value to an event
P : F -> [0,1]
A -> P(A)
(The probability of the entire sample space is 1)
(If A1, A2,… are disjoint in F (mutually exclusive) P(U∞i=1(Ai)) = ∑P(Ai))
Conditional probability
PB : F -> [0,1]
A -> P(A|B)
such that P(A|B) = P(A ∩ B)/P(B)
Continuity of probability measures
Let A1 ⊂ A2 ⊂ … be an expanding series of events
and A = U(An)
then P(A) = limn->oo P(An)
Borel event space
The borel event space over R is the collection of all countable unions and intersections and the complements of open sets
Open sets can be expressed as countable union of closed sets (a,b) = U[a+1/n,b-1/n]
Random variable
A random variable is any fucntion that gives a numerical value to any outcome in a sample space
Inverse image
the set of outcomes which give a value in B such that
X^-1(B) = {w: X(w) in B}
Support
The set of values where the random variable has non-zero probability
For a transformation of a RV g(X) = Y
supp(fY ) = {g(x) : x ∊ supp(fX )}
PDF and CDF of Injective transformations
If Y = g(X) is injective then
PDF: fY (y) = fX[g^-1(y)]|d/dy (g^-1(y)|
CDF: FY(y) = FX[g^-1(y)] if increasing and 1- FX[g^-1(y)] if decreasing
Injective transformations (special case)
if Y = g(x) = a + bX
PDF: fY(y) = 1/|b| fX((y-a)/b)
CDF: FY(y) = FX((y-a)/b) if b>0 and 1 - FX((y-a)/b) if b<0
pseudo-random number
- obtained a uniformly distributed pseudo-random number u ∊ [0,1] such that u = F(x) and x = F^-1(u)
- The number x = F^-1(u) is a pseudo-random number from distribution F
Expectation of probability distributions
For a discrete distribution:
E[X] = ∑ (xi p(xi))
For continuous distributions:
E[X] = ∫ (x f(x))
Variance and Expectation of transformed variables
Var[X] = E[X^2] - E[X]^2
E[g(X)] = ∑ g(xi) f(xi) when X is discrete
E[g(x)] = ∫ g(xi) f(xi) when X is continuous
signed variables and expectation
signed variables exist where the expected values of a function may cancel out where the area under a function may have posotive and negative parts
X = X+ + X-
E[X] = E[X+] + E[X-]
Law of total expectation
E[Y] = E[E[Y|X]
Product moment of X and Y
E[XY] = ∫ ∫ xyF(x,y) dxdy
Covariance of X and Y
Cov[X,Y] = E[XY] - E[X]E[Y]
Correlation coefficient of X and Y
ρ[X,Y] = Cov[X,Y]/√(Var[X]Var[Y])
X and Y independent?
X and Y are independent if
F(X,Y) = F(X)F(Y)
or
f(X,Y) = f(X)f(Y)
Conditional CDF
F(Y|X) = F(X,Y)/F(X)
Conditional PDF
f(Y|X) = f(X,Y)/f(X)
Law of total probability
f(Y) = ∫ f(Y|X) f(X) dx
Law of Total Variance
Var(Y |X) = E(Y^2|X) E(Y |X)^2
such that
Var(Y ) = E[Var(Y |X)] + Var[E(Y |X)]
Conditional expectation
E[Y|X] = ∫ y f(y|x) dy
Power generating functions (PGF)
GX(t) = E[t^X] = ∑ pkt^k
where pk is the probability P(X=k) and t is a real or complex number within the region
Moment generating functions (MGF)
MX(t) = E[exp(tx)]
Central limit theorem
Let X be a random variable with mean µ and variance 𝜎^2
Sn = ∑ Xi
E[Sn] = nµ and Var[Sn] = n𝜎^2
the dist. is
(Sn - E[Sn])/√ Var(Sn) or equi. 1/√n ( ∑ (Xi - µ)/ 𝜎)
standard normal.
CLT distributions
Sn = ∑ Xi ~ N(nµ,n𝜎^2) when n is large mean and variance inc so it loses its shape
Zn = 1/√n ∑ Xi ~ N(√n µ, 𝜎^2) when n is large it keeps shape
X⁻n = 1/n ∑ Xi ~ N( µ, 𝜎^2/n) when n is large loses shape low variance high mean law of large numbers.
Joint likelihood functions
L(θ ; x) = ∏f(xi,θ) where is the PMF or PDF of X
log-likelihood functions
l(θ ; x) = ∑logL(θ ; x) = ∑logf(xi;θ)
take l’(θ) to determine MLE
Baye’s theorem
P(Aj|B) = P(B|Aj)P(Aj)/∑P(B|Ak)P(Ak) where P(B) = ∑P(B|Ak)P(Ak)
Posterior distribution
initial estimate = π0(θ)
posterior dist = π1(θ|X) = f(x|θ)π0(θ)/∫or∑ f(x|θ)π0(θ)
MAP estimate
The MAP estimate is the mode of the posterior distribution such that θ̂ = argmax[π1(θ|X)] where argmax is the value of x opposed to f(x) in max
