QE 1: Probability Theory

Question 1

What are Kolmogorov’s axioms of probability?

Answer 1

(1) P(A) ≥ 0 for all events A.
(2) P(Ω) = 1
(3) P(A or B) = P(A) + P(B) if A and B are mutually exclusive.

Question 2

What is independence? Define three different ways.

Answer 2

Events A and B are independent if any of the following hold:
P(A|B) = P(A)
P(B|A) = P(B)
P(A and B) = P(A)P(B)

Question 3

What does it mean to say that the expectation is a ‘linear operator’? Why is it one?

Answer 3

E[aX+bY] = aE[X] + bE[Y].

Question 4

What is the ‘law of the unconscious statistician’?

Answer 4

E[g(x)] = ∫g(x)f(x)dx

Question 5

What is the law of iterated expectations?

Answer 5

E[E[Y|X]] = E[Y]

Question 6

What is ‘i.i.d data’?

Answer 6

Observations are independently and identically distributed.

Question 7

In what sense is E[Y|X] the ‘best possible predictor for Y on the basis of X’?

Answer 7

E[Y|X] minimises E(Y-m(X))^2.

Question 8

Define convergence in probability.

Answer 8

Xn converges in probability to c, if for every small e > 0, P{|Xn - c|≤e} -> 1 as n grows large.

Question 9

What is the Law of Large Numbers?

Answer 9

The (weak) Law of Large Numbers claims that:

Suppose Xi is i.i.d. with finite mean and variance. Then, the sample mean is a consistent estimator of the population mean.

Question 10

What is the Central Limit Theorem?

Answer 10

If Xi is i.i.d. with finite mean and variance, then √n(xbar - mu) / sigma -> N[0,1].