2P7 Mathematics Flashcards
What is a sample space?
The set of all possible outcomes in an experiment Ω
What are the three axioms of probability?
P(A) >= 0
P(Ω) =1
P(A U B) = P(A) + P(B) for A,B C Ω and A ∩ B = ∅
where ∅ is the empty set
How is the probability A given B defined?
P(A|B) = P(A∩B)/P(B)
How are probability mass functions and cumulative distribution function defined for discrete random variables?
Px(x) = P(X=x)
Fx(x) = P(X<=x)
How is the joint PMF defined for two variables?
P(x,y) = P( X=x ∩ Y=y )
What is Baye’s rule?
P(B|A) = P(A|B)P(B)/P(A)
What is marginilisation for joint PMFs?
Px(x) = sum over y (P(x,y))
What is the law of total probability?
P(A) = P(A|B)P(B) + P(A|B^c) P(B^c) where B^c is the complement of B
Definition of independence of two events.
P(A∩B)=P(A)P(B)
Define expectation
E(g(X)) = sum(g(x)Px(X)) for all x in X, where X is the support of X
It is linear.
What is the expectation rule for two independent random variables?
E(XY) = E(X)E(Y) if X and Y are independent
Define Var[x]
E[(X-E[X])^2] = E[x^2] - E[x]^2
Define entropy
H[X] = E[-log2P(X)]
What is the Bernoulli Distribution?
A trial with binary output
X∈{0,1}, p ∈ [0,1]
X ~ Ber(p)
Pk = { p if k=1, 1-p if k=0, 0 otherwise}
What is the geometric distribution used for?
How many trials until I’m successful. X ~ Geo(p)
Probability of success after x trials on that final trial.
What is the formula for the geometric distrbution?
P(k) = p(1-p)^(k-1)
What does the binomial distribution describe?
How many times was I succesful after n trials?
X~B(n,p) where n is the numper of trials, and p is the probability of successs.
What does the poisson distribution describe?
Ho wmany times was I succesful given a success rate λ
X~Pois(λ),
λ is the average number of events/time interval
Px(k) is the probability that there are k of these events in the time interval.
How does the Bernoulli distribution relate to the binomial?
Xj~Ber(p)
sum over j (Xj) ~ B(n,p)
How does the binomial distribution relate to the poisson distribution?
B(n, λ/n) -> Pois(λ)
What are some properties of the cumulative distrbution function F(x)?
Non decreasing,
Fx(a) < Fx(b)
Limits are 0 and 1 for -∞ and +∞ respectively
Fx(b) - Fx(a) = P (a<= X <= b)
Definition of the joint CDF?
F(x,y) = P (X <= x ∩ Y<=y)
What is the exponential density function?
X~Exp(λ) where λ is the rate of successes, and X is the distance between two successes.
What is the beta density function?
The PDF of the trial probability if we observe α−1 successes and β−1 failures.
X∼Beta(α,β)
X∈[0,1] X is also a probability and has a probability distribution.
If X, and Y are discrete random variables and Y =g(X) how do you extract Py(y)
Py(y) = sum(Px(x)) the sum over x | g(x)=y
If X, and Y are continuous random variables and Y =g(X) how do you extract fy(y)
given g is strictly monotonic,
f(y) = fx(g-1(y))/|g’(g-1(y))|
What is the normalised Gaussian distribution?
(X-µ)/σ ~ N(0,1)
How do you find the distribution of S=X+Y for discrete?
Ps(s) = sum over y ( P(s-y, y) = sum over x (P(x, s-x))
if X and Y are independent:
P(s) = sum over y (Px(s-y) Py(y)) [convolution]
How do you find the distribution of S=X+Y for continuous?
fs(s) = integral ( fxy(s-y, y)dy)
for independent
fs(s) = integral ( fx(s-y) fy(y) dy ) fs = fx*fy where * is convolution
How do expectation and variance variance change with S = X+Y?
E[S] = E[X] + E[y]
Var[s] = Var[X] + Var[Y] + 2Cov[X,Y]
where COV[X,Y] = E[XY] - E[X]E[Y]
if X and Y are indepedent Cov[X,Y] = 0
What is correlation of two random variables?
ρ = Cov[X,Y]/sqrt(Var[X]Var[Y])
What is the probability generating function?
Gx(z) = sum(z^k Px(k)) = E(Z^x)
What’s some properties of the probability generating function for discrete random variables?
What’s the moment generating function for continuous random variables?
gx(s) = integral(fx(x) e^sx ds) = E ( e^sx)
What’s some properties of moment generating function?
What’s the central limit theorem?
If you have a bunch of independent random variables distribution Gaussianly with means µi and variances σi^2
then
[sum(Xi) - sum(µi )]/sqrt(sum(σi^2)) ~ N(0,1)
What is the analogy for variance in a multivariate Gaussian?
Σ where Σij = E[(Xi-µi)(Xj-µj)]
