Paper 1 HPO Flashcards
1
Q
Bayesian optimization
A
Init archive
Loop {
fit surrogate model (u, sigma) on each lambda in archive
build acquisition function from performance and uncertainty
obtain proposed new lambda
eval proposal
add proposal and it’s eval to archive
}
return best performing lambda
2
Q
What surrogate if Lambda is purely real valued?
A
Gaussian process (does not support conditions)
3
Q
What surrogate if there are discrete hyperparameters?
A
Random forest
4
Q
What is Expected Improvement?
A
A popular acquisition function
5
Q
A popular acquisition function?
A
Expected Improvement
6
Q
What is Hyperband?
A
Running Successive Halving multiple times with different starting populations (calling it different brackets)
