Estimators Method of Moments and method of likelihood

Question 1

Explain statistical inference

Answer 1

Statistical inference refers to the process of characterising the properties
of a population, based on the information provided by a realised finite
random sample. Our objective is to find a good estimator of the parameters of interest

Question 2

Explain the method of moments

Answer 2

Because we have a defined probability distribution we will have all the moments of the population in theory if they are defined. The method of moments estimators are found by equating the first k
sample moments (observed) to the corresponding k population moments
(unknown), and solving the resulting system of simultaneous equations

Question 3

How many equations will we need to solve for the method of moments estimator

Answer 3

As many as we have parameters that are unknown.

Question 4

In words what is the likelihood

Answer 4

Note: the likelihood represents how likely each value of θ is, with respect
to the fixed observed data.

Question 5

Describe a statistical model

Answer 5

Triplet consisting of

1. sample space where the observations take values
2. the likelihood for parameter of interest theta - usually an assumption
3. space the parameter take svalues

Question 6

What is the likelihood principle

Answer 6

In a statistical model, all the relevant information regarding
inference on θ is contained in the likelihood function. Also, if the likelihood
values for two realisations of x are proportional for any θ, then they must lead to the same inferential conclusions.

Question 7

How can we use the likelihood to estimate theta

Answer 7

Maximisation of the likelihood to infer theta . This is called the maximum likelihood estimator

Question 8

What transformation do we use to find the maximisation of the likelihood

Answer 8

To find the optimal value of θ we consider the maximisation of the log likelihood. This transformation does not change the position of the max likelihood.

Question 9

How does one find an MLE

Answer 9

We can study the first two derivatives of lx(θ) to check if an analytical
solution is available. Setting the first derivative equal to zero gives a critical point and the second derivative of log likelihood determines the type of point - is it a maximum.
Alos must check the boundaries

Question 10

What if no analyatical solution is available

Answer 10

we resort
to approximations/simplifications (of the likelihood) or heuristic numerical
methods (for the optimisation part), or both!
5

Question 11

How to tell if critical point is a maximum

Answer 11

Second derivative is less than zero

Question 12

How to check the boundaries

Answer 12

Check limit as the parameter approaches infinity to check that these boundary figures are less than the MLE

Question 13

What is our conclusion if boundaries are good and MLE is the max

Answer 13

We conclude that the estimator is a global maximum and hence “” is the MLE