When to use each Algorithm? Flashcards
1
Q
Gradient Ascent/Descent
A
Gradients and smooth functions
2
Q
Newton-Raphson
A
Continous, differential (first and second order)
3
Q
CCS/CCS Accelleration
A
No gradients
4
Q
Powell’s Method
A
No gradients, want faster convergence
5
Q
Hooke’s Jeeves/ GPS
A
Noisy black box problems
6
Q
Nelder-Mead
A
No gradients, smooth functions
7
Q
Hill Climbing
A
Simple local search
8
Q
Steepest Ascent HC
A
Best local improvement (over HC)
9
Q
HC with Replacement
A
For faster sampling compared to HC
10
Q
HC with Restarts
A
To avoid local optima (as HC gets stuck in them)
11
Q
Simulated Annealing
A
Global search, ruggeed landscape
12
Q
Tabu Search
A
When you want to avoid revisits and to avoid issues with hill climb (local minima traps and cycling)
13
Q
Iterated Local Search
A
Strong local search - using homebase and perturb
