Approximation Heuristics

Question 1

Explain research heuristics

Answer 1

Research on heuristics typically focuses on the average empirical behaviour of the algorithms

Question 2

If Theta_hat is an MLE what will the score function be

Answer 2

S(theta_hat)=0

Question 3

If Theta_hat is an MLE what will the observed fisher information be

Answer 3

I(theta_hat)>=0

Question 4

What is the shape of the likelihood function?

Answer 4

It can be approximately gaussian by examining taylor expansion around point theta_hat

Question 5

Name a numerical procedure that can be used to find and approximate the MLE if an analytical solution is not available

Answer 5

Newton-Raphson algorithm

Question 6

Describe the Newton-Raphson algorithm

Answer 6

This is an iterative procedure - each step improves the estimator value
he algorithm requires a starting value, which could for example be the
method of moment estimator

Question 7

By the newton rhapson method how will one know they are getting closer to the maximum

Answer 7

The score function will approach zero as we get closer to the critical
point,