6-ExactOptimisation

Question 1

How do we find optimal point in exact optimisation?

Answer 1

Finding the parameters that maximise / minimise an objective function, requires us to find the points where the derivative of the function is zero

Question 2

Why can we take the log of the likelihood to assess optimality?

Answer 2

Log is a monotonic transformation and therefore preserves optimal points

Question 3

How do we optimise with constraints?

Answer 3

Formulate constraint as an equation equal zero and set to equal g.

Compute lagrangian: L = f - lambda*g

Take derivative of lagrangian

Question 4

What is likelihood in MLE?

Answer 4

Likelihood measures how well a set of model parameters accounts for observed data. In MLE, likelihood measures how probable the observed data is under the assumptions of the model.

In MLE, we want to find the parameters that maximise likelihood:

Theta_MLE = argmax(Likelihood(Theta, D)) = argmax(P(D|theta))