8 | Parameter estimation 2: likelihood-based methods

Question 1

Consider a regression model

y_i = f(z_i;θ) + ε_i.

What are the advantages of using maximum likelihood estimation compared to least squares estimation?

☐ In addition to estimation, the maximum likelihood estimate can be used for model selection.
☐ Maximum likelihood estimation is possible for a larger class of models
☐ Maximum likelihood estimation works beyond regression models, whereas least squares estimation is restricted to regression models.
☐ Maximum likelihood estimation yields a better estimate θ^ML
than least squares estimation.

(ungraded quiz)

Answer 1

In addition to estimation, the maximum likelihood estimate can be used for model selection.

Maximum likelihood estimation is possible for a larger class of models

Question 2

When does p(y;θ,σ2) = ∏_1<=i<=n p(yi;θ,σ2)
hold?

☐ If the errors are normally distributed
☐ If the parameters are normally distributed
☐ If the parameters are independent
☐ If the errors are independent

(ungraded quiz)

Answer 2

If the errors are independent

Question 3

Among the following models, select the ones that are nested:

Question 7Answer

fA(t;θ) = tθ1 and fB(t;θ) = tθ1 / (t+θ2)

fA(t;θ) = tθ1/(t + θ2) and fB(t;θ) = t*θ1^θ3 / (t^θ3+(θ2)^θ3)

fA(t;θ) = θ1+ tθ2 and fB(t;θ) = (θ1+tθ2)^2

fA(t;θ) = θ1e^(−tθ2) and fB(t;θ) = θ1 + θ2e^(−θ3t)

(ungraded quiz)

Answer 3

fA(t;θ) = tθ1/(t + θ2) and fB(t;θ) = t*θ1^θ3 / (t^θ3+(θ2)^θ3)
(we can choose θ=(θ1,θ2,1) in fB to obtain fA)

fA(t;θ) = θ1e^(−tθ2) and fB(t;θ) = θ1 + θ2e^(−θ3t)
( we can choose θ=(0,θ1,θ2) in fB to obtain fA).

Question 4

Assume three models have been estimated using maximum likelihood estimation, as described in the table below. Which one would you choose according to AIC?

Model A Model B Model C
log-likelihood 27 30 31
number of parameters 2 3 5

(ungraded quiz)

Answer 4

AICs:
Model A: 227 -22 = 50
Model B: 230 - 23 = 54
Model C: 2* 31 - 2*5 = 52

–> Model B

Question 5

AIC?

Answer 5

The Akaike information criterion (AIC) of a model f with k
parameters is defined as

AIC = 2 logL(θ^ˆML, σ^ˆ2ML) − 2k