10-Iterative optimisation

Question 1

What is iterative optimisation?

Answer 1

Iterative optimisation is an alternative to exact optimisation. Iterative optimisation is used when there is no closed form solution exists that can be used for exact optimisation

Instead we have to iteratively improve our estimate until we arrive at a satisfactory solution.

Question 2

What is gradient descent?

Answer 2

Gradient descent is a method to do iterative optimisation. It works by taking the partial derivative of each variable and shifting in that direction based on a step size.

Question 3

What is the issue with gradient descent?

Answer 3

Gradient descent, can be stuck at a local minimum if the function is non-convex

Question 4

Why is gradient descent significant?

Answer 4

Gradient descent enables efficient optimisation, when direct methods are infeasible. It allows models to be tuned, e.g. neural networks. It is useful when dealing with large datasets, by processing data in chunks.