Implementing Simulated Annealing
SA is not part of the built-in implementation of the hyperparameter tuning method in Optuna. However, as mentioned in the first section of this chapter, we can define our own custom sampler in Optuna. When creating a custom sampler, we need to create a class that inherits from the BaseSampler class. The most important method that we need to define within our custom class is the sample_relative() method. This method is responsible for sampling the corresponding hyperparameters from the search space based on the hyperparameter tuning algorithm we chose.
The complete custom SimulatedAnnealingSampler() class with geometric cooling annealing schedule (see Chapter 5) has been defined and can be seen in the GitHub repo mentioned in the Technical requirements section. The following code shows only the implementation of the sample_relative() method within the class:
class SimulatedAnnealingSampler(optuna.samplers.BaseSampler):
...
