Lecture 11 - Stochastic Optimisation

Question 1

What are stochastic optimization methods?

Answer 1

Methods that use random variables as part of the optimization.

Question 2

What 2 situations give rise to stochastic optimization?

Answer 2

Optimization of Stochastic functions - objective functions which contain random noise.
Optimization algorithms with random search methods - search for the optimum using na algorithm that contains an element of randomness.

Question 3

What are the consequences of stochasticity in objective functions?

Answer 3

Evaluation of objective function is no longer accurate

Evaluation of gradient of objective function no longer accurate

Question 4

Why are stochastic search algorithms needed?

Answer 4

Can inject randomness into the search process which can help speed convergence and improve robustness in cases where the global optimum is hard to find.