Tuning hyperparameters

All machine learning algorithms have hyper-parameters or settings that can change how they operate. These hyper-parameters can improve the accuracy of a model or reduce the training time. We have seen some of these hyper-parameters in previous chapters, particularly Chapter 3, Deep Learning Fundamentals, where we looked at the hyper-parameters that can be set in the mx.model.FeedForward.create function. The techniques in this section can help us find better values for the hyper-parameters.

Selecting hyper-parameters is not a magic bullet; if the raw data quality is poor or if there is not enough data to support training, then tuning hyper-parameters will only get you so far. In these cases, either acquiring additional variables/features that can be used as predictors and/or additional cases may be required.

Grid search

For more information on tuning hyper-parameters, see Bengio, Y. (2012), particularly Section 3, Hyperparameters, which discusses the selection and characteristics...