4. Expected Values

Question 1

Expectation

Discrete Case

Answer 1

E(X) = Σ xi*p(xi)
-limitation, if it’s a finite sum and the xi are both positive and negative, the sum could fail to converge, we restrict to cases where the sum converges absolutely:
Σ |xi| p(xi) < ∞
-otherwise the expectation is undefined

Question 2

Expectation

Continuous Case

Answer 2

E(X) = ∫ x*f(x) dx

• integral between -∞ and +∞
• if ∫ x*f(x) dx = ∞ we say that the expectation is undefined

Question 3

Expectation of Gamma Distribution

Answer 3

E(X) = α/λ

Question 4

Expectation of Exponential Distribution

Answer 4

-the exponential distribution is just the gamma distribution with parameter α=1
E(X) = 1/λ

Question 5

Expectation of Normal Distribution

Answer 5

E(X) = µ

Question 6

Expectations of Functions of Random Variables

Theorem A - Discrete Case

Answer 6

-let g(x) be a fixed function
E(g(x)) = Σ g(xi) p(xi)
-with limitation Σ g(xi) p(xi) < ∞

Question 7

Expectations of Functions of Random Variables

Theorem A - Continuous Case

Answer 7

-let g(x) be a fixed function
E(g(x)) = ∫ g(x)*f(x) dx
-with limitation Σ g(x) f(x) < ∞

Question 8

Expectations of Functions of Random Variables

Theorem B - Discrete Case

Answer 8

-suppose X1,X2,…,Xn are jointly distributed random variables, let Y=g(X1,…,Xn)
E(Y) = Σ g(x1,…,xn) p(x1,…,xn)
-with:
Σ |g(x1,…,xn)| p(x1,…,xn) < ∞

Question 9

Expectations of Functions of Random Variables

Theorem B - Continuous Case

Answer 9

-suppose X1,X2,…,Xn are jointly distributed random variables, let Y=g(X1,…,Xn)
E(Y) = ∫…∫ g(x1,…,xn) f(x1,…,xn) dx1….dxn
-with:
∫…∫ |g(x1,…,xn)| f(x1,…,xn) dx1…dxn < ∞

Question 10

Expectations of Functions of Random Variables

Theorem C

Answer 10

-suppose X1,…,Xn are jointly distributed random variables with expectation E(Xi) and Y = a + Σ biXi
-then:
E(Y) = a + Σ biE(Xi)

Question 11

Variance

Definition

Answer 11

-the variance of a random variable X is defined as :

Var(X) = E{X-E(X)}²

Question 12

Standard Deviation

Definition

Answer 12

σ = √Var(X)

Question 13

Variance

Y= a + bX Theorem

Answer 13

-if Y = a + bX, then:

Var(Y) = b²Var(X)

Question 14

Variance

Alternative Form Theorem

Answer 14

-the variance of X if it exists may also be computed by:

Var(X) = E(X²) - (E(X))²

Question 15

Covariance

Definition

Answer 15

-for jointly continuous random variables X and Y, we define the covariance of X and Y as:
Cov(X,Y) = E(XY) - E(X)E(Y)

Question 16

Relationship Between Variance and Covariance

Answer 16

-variance is just the covariance of a random variable with itself:
Var(X) = Cov(X,X)

Question 17

Correlation Coefficient

Definition

Answer 17

-we define the correlation coefficient of X and Y:
ρ = Cor(X,Y) = Cov(X,Y) / √[Var(X)Var(Y)]
-it measures the linear relationship between two random variables and takes values:
ρ ≤ 1

Question 18

Variance of the Sum of Random Variables

Answer 18

-suppose that Xi is a sequence of random variables with joint pdf:
f(x_) = f(x1,x2,...,xn)
-we have:
var(ΣXi) = Σ var(Xi) + 2 Σ cov(Xi,Xj)
-where the first sum if from i=1 to i=N
-and the second is for all i

Question 19

Independence and Covariance

Answer 19

-suppose X and Y are independent random variables, then:
Cov(X,Y) = 0
-BUT zero covariance does not generally imply independence!!

Question 20

Covariance of the Sum

Answer 20

-suppose:
U = a + ΣbiXi
-and
V =c + ΣdjYj
-then the covariance :
Cov(U,V) = ΣΣbidjcov(Xi,Yj)
-where the sums are from i=1 to i=n, and j=1 to j=m

Question 21

Moment

Definition

Answer 21

-suppose X is a random variable with pdf f, define:
E(X^n) = ∫ x^n f(x) dx
-where the integral is from -∞ to +∞
-we call E(X^n) the nth moment of X

Question 22

Variance

moment Definition

Answer 22

Var(X) = second moment - (first moment)²

Question 23

Moment Generating Function

Definition

Answer 23

-suppose X is a random variable with pdf f
-let M(t) = E(e^(tX))
M(t) = ∫ e^(tX) f(x) dx
-where the integral is from -∞ to +∞
-we call M(t) the generating function because:
∂^n M(t) / dt^n |0 = E(X^n)
-i.e. the nth derivative of M(t) with respect to n evaluated at t=0 is the expectation of X to the power n, or the nth moment of X

Question 24

Two Functions With the Same Generating Function

Answer 24

-if two functions have the same moment generating function Fx(x) = Fy(y) for ALMOST all x

Question 25

Moments of Sums of Independent Random Variables

Answer 25

-suppose Xi are a sequence of independent random variables, define:
Y = Σ Xi
-then:
My(t) = ∏ Mxi(t)

Question 26

Facts About Moment Generating Functions

Answer 26

M_aY = M_Y(at)
M_(Y+a) = e^(at) M_Y(t)