245 Proofs

Question 1

Efficiency definition

Answer 1

Given two unbiased estimators θˆ 1 and θˆ 2 of a parameter θ, with variances V(θˆ 1) and V(θˆ 2), respectively, then the efficiency of θˆ 1 relative to θˆ 2, denoted eff (θˆ 1, θˆ 2), is defined to be the ratio:

eff(θ^ 1, θ^ 2) = V ( θˆ 2 ) / V ( θˆ 1 )
.

Question 2

Likelihood definition

Answer 2

Suppose that the likelihood function depends on k parameters θ1, θ2, …, θk.

Choose as estimates those values of the parameters that maximize the likelihood:

L(y1, y2,…, yn |θ1,θ2,…,θk).

Question 3

Invariance Property of MLE

Answer 3

If θ is the paramter associated with a distribution of some random variable Y, then for any function t(θ), where the maximum likelihood estimator of θ is θ^, then the MLE of t(θ) will be given by (t(θ))^ = t(θ^)

Question 4

Power defintion

Answer 4

Suppose that W is the test statistic and RR is the rejection region for a test of a hypothesis involving the value of a parameter θ. Then the power of the test, denoted by power(θ), is the probability that the test will lead to rejection of H0 when the actual parameter value is θ. That is,
power(θ) = P(W in RR when the parameter value is θ).

Question 5

Neyman-Pearson Lemma

Answer 5

Suppose that we wish to test the simple null hypothesis H0 : θ = θ0 versus the simple alternative hypothesis Ha : θ = θa , based on a random sample Y1,Y2,…,Yn from a distribution with parameter θ.

Let L(θ) denote the likelihood of the sample when the value of the parameter is θ. Then, for a given α, the test that maximizes the power at θa has a rejection region, RR, determined by

LR = L(θ0)/L(θa) < k

Now any other test that has a significance level less or equal to a, will have a power less or equal to the likelihood ratio test LR

Question 6

Joint Distribution for n=2

Answer 6

Proof

Question 7

Generalised Likelihood Ratio test

Answer 7

Definition

Question 8

Conjugate Prior Distributions

Answer 8

Let L be a likelihood function (θ|x) . A class Π of prior distributions is said to form a conjugate family if the posterior density

p(θ|x) /= p(θ)(θ|x)

is in the class Π for all x whenever the prior density is in Π.

Question 9

Suppose that θ ~ N(θ0,Φ0) and that X = (X1,…,Xn), where the Xi are independent and Xi ~ N(θ,Φ). Xi | θ ~ N (q , f ).

2.2

Answer 9

Definition

Question 10

Remark 1 of Thm 2.2

Answer 10

Proof

Question 11

Remark 2 of Thm 2.2

Answer 11

Proof

Question 12

Remark 3 of Thm 2.2

Answer 12

Proof

Question 13

Remark 4 of Thm 2.2

Answer 13

The posterior distribution can be used to calculate probabilities concerning θ. For a given probability 1- a, the interval A = [a1,a2] can be calculated such that P(θ an element of A) = 1- a. Alternatively, for a given interval B = [b1,b2], the probability b = P(θ an element of B), can be found.

Question 14

Suppose that Φ ~ Inv-G(a,b) and that X = (X1,…,Xn), where the Xi’s are independent and Xi |Φ ~ N(μ,Φ).

2.3

Answer 14

Proof and Definition

Question 15

Remark 1 Thm 2.3

Answer 15

Proof

Question 16

Remark 2 Thm 2.3

Answer 16

Definition

Question 17

Suppose that π ~ Beta(a,b) and that you have an observation X such that X ~ Bin(n,p).

3.3

Answer 17

Definition and Proof

Question 18

Remark 1 Thm 3.3

Answer 18

Statement

Question 19

Remark 2 Thm 3.3

Answer 19

Proof

Question 20

Remark 3 Thm 3.3

Answer 20

Proof

Question 21

Remark 4 Thm 3.3

Answer 21

Proof

Question 22

Suppose that λ ~ G(a, 1/b) and that X = (X1,…,Xn), where the Xi’s are independent and Xi|λ ~ Poiss

3.4

Answer 22

Defintion and Proof

Question 23

Remark 1 of 3.4

Answer 23

Definition

Question 24

Suppose that θ ~ Pa(ξ ,γ) and that X = (X1,…,Xn) where the Xi’s are independent and
Xi | θ ~ U(0, θ).

3.5

Answer 24

Definition and Proof

Question 25

Suppose that p(θ) = c, -∞ < q < ∞ and that X = (X1,…,Xn), where the Xi’s are independent and Xi | θ ~ N (θ , Φ)

4.1

Answer 25

Definition

Question 26

Suppose that p(Φ) = 1/Φ and that X = (X1,…,Xn), where the Xi’s are independent and Xi|Φ⇠N(μ,Φ)

4.2

Answer 26

Definition

Question 27

Suppose that π ~ Beta(0,0) and that you have an observation X such that X ~ Bin(n, π).

4.4

Answer 27

Definition

Question 28

Suppose that p(λ) /= λ^-1/2 and that X = (X1,…,Xn) where the Xi’s are independent and Xi|λ~Poiss(λ)

4.5

Answer 28

Definition

Question 29

Suppose that θ ~ Pa(0,0) and that X = (X1,…,Xn), where the Xi are independent and Xi | θ ~ U (0, θ).

4.6

Answer 29

Definition

Question 30

Loss Function without observations

Answer 30

Definition

Question 31

Loss functions with observations

Answer 31

Definition

Question 32

Bayes Estimator

Answer 32

Suppose now that the value x of the random vector X can be observed before estimating θ, and let p (θ | x) denote the posterior ff or df of θ . For anyZ estimate a, the expected loss in this case will be

E[l(θ,a)|x] = Integral of l(θ,a)p(θ|x)dq

Here an estimate a should now be chosen for which this expectation is a minimum.

For each possible value x of the random vector X , let δ(x) denote a value of the estimate a for which the expected loss in equation 4.1 is a minimum. Then the function δ(X) for which the values are specified in this way will be an estimator of θ. This estimator is called a Bayes estimator of θ. In words, for each possible value x in X , the value δ(x) of the Bayes estimator is chosen so that

E[l(θ,δ(x))|x]=minE[l(θ,a)|x].

Question 33

Squared Error Loss Function

Answer 33

Definition

Question 34

E[(θ-a)^2)|x] is a minimum when a is chosen to be equal to the mean of the posterior distribution

Answer 34

Proof

Question 35

Absolute Error Loss Function

Answer 35

Definition

Question 36

E[|θ-a||x] is a minimum when a is chosen to be equal to the median of the posterior or equals F(θ|x)^-1(1/2)

Answer 36

Proof