Conditional expectation

Question 1

Loss function
▪ Definition

Answer 1

Consider two random variables, X and Y.
Assume:
▪ X is Rⁿ-value
▪ Y is real valued

The idea is that X is observable and Y has to be “predicted” on the basis of X

We consider the following
“Best prediction problem”
find a function
Ϥ: Rⁿ –> R
such that the “loss function” is minimized

L(Ϥ) = E[(Y-Ϥ(X))²] –> least squared
loss function

Question 2

Conditional density
▪ Definition

Answer 2

We assume (X,Y) is a discrete random vector, with joint density pₓ,y

p y|x (y|x) = P(Y=y|X=x)
p y|x (y|x) = (P(Y=y|X=x)) / P(X=x)
p y|x (y|x) = pₓ,y (x,y) / pₓ(x)

Question 3

Conditional expectation
▪ Definition

Answer 3

We define the conditional expectation of Y given X=x as follows

E(Y|X=x) = Σ y py|ₓ (y|x)

More generally

E(g(x,y)|X=x) = Σ g(x, y) py|ₓ (y|x)

Question 4

Conditional expectation
▪ Properties

Answer 4

1) E[aY+bZ|X=x] = aE(Y|X=x) + bE(Z|X=x)
2) E[f(x)|X=x] = f(x)
3) E[f(x)Y|X=x] = f(x) * E[Y|X=x]
4) E[ E(Y|X) ] = E(Y)

Question 5

Conditional expectation
▪ Theorem

Answer 5

The function Ϥ(x) = E(Y|X=x) solves the best prediction problem

In other words, for any
g:Rⁿ–>R

L(g) ⩾ L(Ϥ)