Imagine a sound system that has a high quality speaker and mixer system. You must have seen a series of buttons on the console that independently control a specific parameter of sound quality. The bass, treble, and loudness are some of the controls that need to be properly set for a great experience. Similarly, a deep neural network is only as good as the setting of various controlling parameters. These parameters are called hyperparameters, and the process of controlling various parameters at a value that gets the best performance in terms of training/execution time as well as accuracy and generalization of the model. Similar to the sound equalizer example, multiple hyperparameters need to be tuned together for optimum performance. There are two strategies typically used when choosing a combination of hyperparameters:
- Grid search: The hyperparameters are...
