Hyperparameter Optimization Flashcards
(True or false) hyperparameter optimization (HPO) is a black-box optimization problem.
True
R package that provides different tuning approaches
mlr3 tuning
The relevant parameter set can be constructed / specified using the ___ command.
ps()
All the details for the tuning procedure need to be combined in a ____ .
Tuning instance
It specifies the method that is used for optimizing the tuning instance.
Tuner
It defines the number of (equidistant) values per configurable parameter that are to be tried during the tuning procedure.
Resolution
To avoid overfitting, tuning itself should be performed during the ___.
Training procedure
A mixture of learner and tuner
Autotuner
(True or false) “irace” returns the elite configurations and can be used for tuning.
True
Finding a good hyperparameter configuration for
the problem at hand.
Hyperparameter optimization
Which are the two main types of baseline optimizers?
Grid search and random search
Baseline in which a number of values is selected for each parameter and evaluate all possible combinations.
Grid search
Baseline in which each parameter is sampled uniformly at random.
Random search
Evaluating novel configurations (very) different to previous ones.
Exploration
Two types of stochastic approaches for hyperparameter optimization.
Simulated Annealing and CMA-ES
Two types of model-based approaches for hyperparameter optimization.
iterated F-race (irace) and Hyperband.
Trying to improve existing configurations by evaluating similar ones.
Exploitation
True or false? In the context of Bayesian optimization, the „expected improvement“ acquisition function trades off exploration and exploitation of the search space. Therefore, it is a suitable method in situations where functions are expensive to evaluate, e.g. hyperparameter configurations in large-scale problems.
True
Another name for Bayesian Optimization
Sequential Model-Based Optimization, SMBO
A fast-to-evaluate model of performance function, based on
already evaluated configurations, with uncertainty estimate.
Surrogate model
Optimization which evaluates more configurations with lower budgets speeds up HPO by terminating bad configurations early, and thus allows better optimization with limited overall budget.
Multifidelity optimization
Type of multifidelity optimization that starts all configurations with a certain fraction of the budget. Then, discards half of the configurations with worst performance, doubles the budget and repeats until final budget is reached.
Succesive Halving (SH)
The three most important parameters of succesive halving.
Budget factor, final budget, and total used budget.
This algorithm tries to alleviate the problem of how
well the initial training phase of SH captures final performance by introducing multiple brackets.
Hyperband
