Defintions Flashcards
Statistic Def
Let J be an arbitrary set.
A function (T: Omega -> J ) independent of Theta is called a statistic.
Statistic Sufficiency
A statistic (T: Omega -> J) is sufficient for Theta if the partition
{ x e Omega : T(x) = a }, a e J
Neymann Factorisation Theorem
A statistic T is sufficient for Theta, if and only if there exists g and h such that,
L(x|Theta) = g(Theta, T(x)) h(x)
Statistic T completeness
A statistic T is complete if its family of distributions is complete.
Exponential Family Completeness Theorem
X is an IID sample from a probability model
P(x|Theta) = exp[ThetaB(x) + C(Theta) + D(x)] , Theta in THETA
If THETA contains an open interval T = sigma B(x) is complete
Bahadurs Theorem
If a statistic T, is sufficient and complete for Theta, it is also minimal sufficient. However, the converse is not true.
MLE
The MLE Theta_hat(x) of Theta, is a value of Theta which maximises the likelihood w.r.t theta for fixed x
Small Volatility(MSE)
We want estimator to have small MSE
MSE = Var[g_hat(X)] + [b_g_hat(Theta)]^2
Consistency of an estimator
Estimator g_hat(x) is consistent if,
G_hat(x) -> g(Theta) as n -> infinity
MLE and sufficiency Def
If T(x) is sufficient in likelihood model, MLE is a function of the sufficient statistic
MLE and CRLB Theorem
If theta_tilde, an unbiased estimator of Theta, attains the CRLM, the theta_tilde is the unique MLE of Theta
Rao-Blackwell Theorem
Let g_hat(x) be an unbiased estimator of g_theta. If T = T(x) is sufficient for Theta then, the estimator g_tilde(t) is an unbiased estimator for g_theta.
G_tilde(t) = E[g_hat(x)|T(x) = t) and
Var(g_tilde(t)) <= Var(g_hat(x))
Lehman-Scheffe Theorem
Suppose there is an unbiased estimator g_hat(x) for g(theta) and a statistic T = T(x) that is sufficient and complete for Theta.
Then,
G_tilde(t) = E[g_hat(x)|T(x) = t] is the unique MVUE of g(theta)
Type 1 and Type 2 errors, size and power
T1 - Reject H0 when H0 is true
T2 - Do not reject H0 when H0 is false
Size - prob(Type 1 Error)
Power - 1 - (Type 2 Error) i.e Reject H0 when false
(UMP test)
A test of size alpha is UMP if,
For all theta in THETA1, its power is greater than or equal to the power of any other test of size alpha
UMP Lemma
There exists a UMP test of size alpha for H0 vs H1 iff there is a test phi that is the MP test of H0 vs H1’: theta = theta1 for all theta in THETA1
3 Examples of Non-Informative Priors and explain them
Ignorant: All values of theta are equally likely
Vague: Choose a prior with a very flat curve e.g gamma then let a,b -> 0.
Jeffrey’s Prior:
Pi(theta) is proportional to the sqrt of Fisher Info
Sufficiency Theorem for Statistic in Posterior
If T(X) is sufficient for theta, then
Pi(Theta|X) = Pi(Theta|T(X) = t)
Conjugate Family Def
A class C of distributions is said to from a conjugate family for the likelihood of the posterior distribution is in C whenever the prior distribution is.
Bayesian Credible Region
The region C(alpha) is a 100(1-alpha)% BCR for theta if
P(Theta e C(alpha)|x) = 1- alpha
Highest Posterior Density Region Def
The region C(alpha) is a HPDR for theta if,
C(alpha) = {Theta: Pi(Theta|x) >= gamma), where gamma is chosen so that
P(Theta e C(alpha)|x) = 1-alpha
HPDR in practice
- Posterior pdf is 0 outside interval (x,y)
- Positive/negative on the interval (x,y)
- Increasing/ Decreasing (f’(x)>0 etc) on interval
Increasing -> (a,y)
Decreasing -> (x,a)
Then integrate the posterior pdf on this interval
Conjugate prior for sigma known, mu unknown
The conjugate prior for mu is N(m,w). And mu is proportional to the N(m, Vsigma^2).
Conjugate prior mu known, signal unknown
When mu is known , the conjugate prior for sigma^2 is the Inverse Gamma(a,d).
Proportional to the IG([a+(mu-m)^2]/v, d + 1) distribution
Conjugate prior for general case when sigma and mu are both unknown
The Normal Inverse Gamma distribution.
NIG(a,d,m,v)
Mu and sigma sq are not independent.
