Defintions

Question 1

Statistic Def

Answer 1

Let J be an arbitrary set.
A function (T: Omega -> J ) independent of Theta is called a statistic.

Question 2

Statistic Sufficiency

Answer 2

A statistic (T: Omega -> J) is sufficient for Theta if the partition
{ x e Omega : T(x) = a }, a e J

Question 3

Neymann Factorisation Theorem

Answer 3

A statistic T is sufficient for Theta, if and only if there exists g and h such that,

L(x|Theta) = g(Theta, T(x)) h(x)

Question 4

Statistic T completeness

Answer 4

A statistic T is complete if its family of distributions is complete.

Question 5

Exponential Family Completeness Theorem

Answer 5

X is an IID sample from a probability model

P(x|Theta) = exp[ThetaB(x) + C(Theta) + D(x)] , Theta in THETA

If THETA contains an open interval T = sigma B(x) is complete

Question 6

Bahadurs Theorem

Answer 6

If a statistic T, is sufficient and complete for Theta, it is also minimal sufficient. However, the converse is not true.

Question 7

MLE

Answer 7

The MLE Theta_hat(x) of Theta, is a value of Theta which maximises the likelihood w.r.t theta for fixed x

Question 8

Small Volatility(MSE)

Answer 8

We want estimator to have small MSE

MSE = Var[g_hat(X)] + [b_g_hat(Theta)]^2

Question 9

Consistency of an estimator

Answer 9

Estimator g_hat(x) is consistent if,

G_hat(x) -> g(Theta) as n -> infinity

Question 10

MLE and sufficiency Def

Answer 10

If T(x) is sufficient in likelihood model, MLE is a function of the sufficient statistic

Question 11

MLE and CRLB Theorem

Answer 11

If theta_tilde, an unbiased estimator of Theta, attains the CRLM, the theta_tilde is the unique MLE of Theta

Question 12

Rao-Blackwell Theorem

Answer 12

Let g_hat(x) be an unbiased estimator of g_theta. If T = T(x) is sufficient for Theta then, the estimator g_tilde(t) is an unbiased estimator for g_theta.

G_tilde(t) = E[g_hat(x)|T(x) = t) and

Var(g_tilde(t)) <= Var(g_hat(x))

Question 13

Lehman-Scheffe Theorem

Answer 13

Suppose there is an unbiased estimator g_hat(x) for g(theta) and a statistic T = T(x) that is sufficient and complete for Theta.
Then,

G_tilde(t) = E[g_hat(x)|T(x) = t] is the unique MVUE of g(theta)

Question 14

Type 1 and Type 2 errors, size and power

Answer 14

T1 - Reject H0 when H0 is true
T2 - Do not reject H0 when H0 is false

Size - prob(Type 1 Error)
Power - 1 - (Type 2 Error) i.e Reject H0 when false

Question 15

(UMP test)

Answer 15

A test of size alpha is UMP if,

For all theta in THETA1, its power is greater than or equal to the power of any other test of size alpha

Question 16

UMP Lemma

Answer 16

There exists a UMP test of size alpha for H0 vs H1 iff there is a test phi that is the MP test of H0 vs H1’: theta = theta1 for all theta in THETA1

Question 17

3 Examples of Non-Informative Priors and explain them

Answer 17

Ignorant: All values of theta are equally likely

Vague: Choose a prior with a very flat curve e.g gamma then let a,b -> 0.

Jeffrey’s Prior:

Pi(theta) is proportional to the sqrt of Fisher Info

Question 18

Sufficiency Theorem for Statistic in Posterior

Answer 18

If T(X) is sufficient for theta, then

Pi(Theta|X) = Pi(Theta|T(X) = t)

Question 19

Conjugate Family Def

Answer 19

A class C of distributions is said to from a conjugate family for the likelihood of the posterior distribution is in C whenever the prior distribution is.

Question 20

Bayesian Credible Region

Answer 20

The region C(alpha) is a 100(1-alpha)% BCR for theta if

P(Theta e C(alpha)|x) = 1- alpha

Question 21

Highest Posterior Density Region Def

Answer 21

The region C(alpha) is a HPDR for theta if,

C(alpha) = {Theta: Pi(Theta|x) >= gamma), where gamma is chosen so that

P(Theta e C(alpha)|x) = 1-alpha

Question 22

HPDR in practice

Answer 22

• Posterior pdf is 0 outside interval (x,y)
• Positive/negative on the interval (x,y)
• Increasing/ Decreasing (f’(x)>0 etc) on interval

Increasing -> (a,y)
Decreasing -> (x,a)

Then integrate the posterior pdf on this interval

Question 23

Conjugate prior for sigma known, mu unknown

Answer 23

The conjugate prior for mu is N(m,w). And mu is proportional to the N(m, Vsigma^2).

Question 24

Conjugate prior mu known, signal unknown

Answer 24

When mu is known , the conjugate prior for sigma^2 is the Inverse Gamma(a,d).

Proportional to the IG([a+(mu-m)^2]/v, d + 1) distribution

Question 25

Conjugate prior for general case when sigma and mu are both unknown

Answer 25

The Normal Inverse Gamma distribution.

NIG(a,d,m,v)

Mu and sigma sq are not independent.