Implementing Grid Search

Implementing Grid Search in Optuna is a bit different from implementing TPE and Random Search. Here, we need to also define the search space object and pass it to optuna.samplers.GridSampler(). The search space object is just a Python dictionary data structure consisting of hyperparameters’ names as the keys and the possible values of the corresponding hyperparameter as the dictionary’s values. GridSampler will stop the hyperparameter tuning process if all of the combinations in the search space have already been evaluated, even though the number of trials, n_trials, passed to the optimize() method has not been reached yet. Furthermore, GridSampler will only get the value stated in the search space no matter the range we pass to the sampling distribution methods, such as suggest_categorical, suggest_discrete_uniform, suggest_int, and suggest_float.

The following code shows how to perform Grid Search in Optuna. The overall procedure to implement...