Probability Flashcards
Definition of the independence of 2 random variables x and y.
x and y are independent if their probability distribution can be expressed as a product of two factors, one only involving x and the second only involving y.
∀x ∈ x, y ∈ y,
p(x = x, y = y) = p(x = x)p(y = y)
x⊥y means that x and y are independent
Definition of x and y being conditionally independent given z.
∀x ∈ x, y ∈ y, z ∈ z,
p(x = x, y = y | z = z) = p(x = x | z = z)p(y = y | z = z).
x⊥y | z means that x
and y are conditionally independent given z.
The expectation, or expected value, of some function f(x) with respect to a probability distribution P(x) is …
…the average, or mean value, that f takes on when
x is drawn from P.
Formula of the expected value of some function f(x) with respect to a probability distribution P(x), when x DISCRETE.
Ex∼P [f(x)] =Sum over x( P (x)f(x) )
Formula of the expected value of some function f(x) with respect to a probability distribution P(x), when x CONTINUOUS.
Ex∼p[f(x)] =integral of p(x)f(x)dx
Expectations are … therefore Ex[αf(x) + βg(x)] = …
linear
αEx[f(x)] + βEx [g(x)]
The variance gives a measure of …
+ Formula
how much the values of a function of a random variable x vary as we sample different values of x from its probability distribution:
Var(f(x)) = E[ (f(x) − E[f(x)])^2 ]
The covariance gives some sense of …
High absolute values of the covariance mean that …
+ Formula
The covariance gives some sense of how much two values are linearly related to each other, as well as the scale of these variables.
High absolute values of the covariance mean that the values change very much
and are both far from their respective means at the same time.
Cov(f(x), g(y)) = E[ (f(x) − E[f(x)]) (g(y) - E[g(y)]) ]
Correlation is different than covariance because it …
normalizes the contribution of each variable in order to measure only how much the variables are related, not giving a shit about their respective scale.
How are covariance and independence related.
Because that are independent will always have zero covariance, and two variables with non zero covariance will be dependent.
How are covariance and independence different.
For 2 variables to have 0 covariance there must be no linear dependence between them, independence is stronger as it also excludes non linear relationships.
What are the 2 main kind of structured probabilistic models or graphical models? And what they are used for?
They are a way of describing a probability distribution. Directed models use the factorization in conditional probabilities and undirected represent factorization in functions
