Implementing Bayesian Optimization Hyper Band
Bayesian Optimization Hyper Band (BOHB) is one of the variants of the Multi-Fidelity Optimization hyperparameter tuning group (see Chapter 6) that the NNI package can implement. Note that to use BOHB in NNI, we need to install additional dependencies using the following command:
pip install "nni[BOHB]"
Let’s use the same data, pipeline, and hyperparameter space as in the example in the previous section to implement BOHB with NNI using pure Python code.
The following code shows how to implement Hyper Band with the NNI package using pure Python code. You can find the more detailed code in the GitHub repository mentioned in the Technical requirements section:
- Prepare the model to be tuned in a script. We’ll use the same
model_advisor.py
script as in the previous section. - Define the hyperparameter space in the form of a Python dictionary. We’ll use the same hyperparameter space as in the...