Grid Search Optimization
The most naïve approach to find the optimal set of hyperparameters for a model would be to use brute-force methods and iterate with every combination of values for the hyperparameters and then find the most optimal combination. This will deliver the desired results, but not in the desired time. In most cases, the models we train will be significantly large and require heavy compute time for training. Iterating through each combination wouldn't be an ideal option. To improve upon the brute-force method, we have grid search optimization; as the name has already indicated, here, we define a grid of values that will be used for an exhaustive combination of values of hyperparameters to iterate.
In layman's terms, for grid search optimization, we define a finite set of values for each hyperparameter that we would be interested in optimizing for the model. The model is then trained for exhaustive combinations of all possible hyperparameter values and the combination with...