Optimality in Estimation

Question 1

Define the risk function

Answer 1

R(delta, theta) = E[ L(delta(X), theta) ] = integral_X L(delta(x), theta) f_theta(x) dx

Question 2

What is the risk function for the mean square error loss function L(a,theta) = (a-theta)^2 ?

Answer 2

R(delta, theta) = MSE(delta)

Question 3

What are the two main applications of decision theory?

Answer 3

• estimation: delta(X) = estimator of theta(X)
• hypothesis testing: delta(X) is a test using the binary loss function 1{delta =/= theta} and R(delta, theta) is the probability of making type I or II error

Question 4

Define inadmissible

Answer 4

there exists a delta*(X) st

• R(delta*, theta) <= R(delta, theta) for all theta
• and strict inequality for some theta

Question 5

Define a pi-Bayes risk and a pi-Bayes decision rule

Answer 5

• R_pi(delta) = integral_Theta R(delta, theta) pi(theta) dtheta = integral_Theta integral_X L(delta(x), theta) f_theta(x) dx pi(theta) dtheta
• delta_pi is any decision rule minimizing R_pi(delta)

Question 6

Define the posterior risk function

Answer 6

R(delta, theta) = E[ L(delta(X), theta) | X] = integral_X L(delta(x), theta) pi(theta | x) dx
for posterior distribution pi(theta | x)

Question 7

2 strategies to find the pi-Bayes decision rule:

Answer 7

• Find the minimizer of the posterior risk because if delta minimizes the posterior risk then it also minimizes the Bayes risk
• directly minimize Bayes risk

Question 8

What is the minimizing decision rule of posterior risk for:

• mean squared error loss?
• absolute value?

Answer 8

• posterior mean

- posterior median

Question 9

2 strategies for providing admissibility

Answer 9

• show estimator is the unique Bayes estimator for some prior pi
• from scratch (eg. by contradiction)

Question 10

Define minimax risk

Answer 10

inf_delta sup_theta R(delta,theta)

Question 11

3 situations when minimax risk is atteined

Answer 11

• Bayes rule with constant risk
• Bayes rule whose Bayes risk R_pi(delta_pi) = sup_theta R(delta_pi, theta)
  (uniqueness of delta_pi => uniqueness of minimax)
• admissible estimator delta with constant risk

Question 12

Define UMVUE

Answer 12

UMVUE g^ is an unbiased estimator of g(theta) such that:

for all theta: var (g^) <= var (g~) for any other unbiased g~

Question 13

Relation between complete and minimal statistic T(x)

Answer 13

T(x) sufficient and complete => T(x) minimal

Question 14

State Basu’s theorem

Answer 14

if T complete => any ancillary statistic V is independent of T
(ancillary statistics has distribution that doesn’t depend on theta)

Question 15

State Lehman-Scheffe theorem

Answer 15

if T sufficient and complete and g~ an unbiased estimator of g(theta)
then g^(T(X)) = E[g~(X)|T(X)] is the UMVUE of g(theta)

Question 16

Strategies to find a UMVUE

Answer 16

If T complete and sufficient:

• g^(T(X)) = E[g~(X)|T(X)] for unbiased g~(theta)
• h(T) unbiased is unique UMVUE

else:
- an estimator that atteins CR lower bound for all theta

Question 17

Link UMVUE and Cramer Rao lower bound

Answer 17

• if UMVUE exists and if CR bound is atteined then UMVUE atteins CR bound
• same for uniformely atteining CR bound