Lecture 11 - Stochastic Optimisation Flashcards
What are stochastic optimization methods?
Methods that use random variables as part of the optimization.
What 2 situations give rise to stochastic optimization?
Optimization of Stochastic functions - objective functions which contain random noise.
Optimization algorithms with random search methods - search for the optimum using na algorithm that contains an element of randomness.
What are the consequences of stochasticity in objective functions?
Evaluation of objective function is no longer accurate
Evaluation of gradient of objective function no longer accurate
Why are stochastic search algorithms needed?
Can inject randomness into the search process which can help speed convergence and improve robustness in cases where the global optimum is hard to find.