Model optimization with grid search
Choosing an algorithm for building a predictive model is not enough. Many algorithms have hyperparameters whose values have a direct impact on the predictive power of the model. So, how do you choose values for your hyperparameters?
A brute-force method would create a grid of all possible values and search over them. This approach is computationally expensive, takes an inordinate amount of time, and ultimately, yields results that are not much better than what we could achieve by other means. We have outlined a strategy for grid search that has proved effective in building optimized models while running in a reasonable amount of time.
The general strategy entails, first, tuning a few key parameters using a Cartesian grid search. These key parameters are those we expect to have the biggest impact on the results. Then, we fine-tune other parameters using a random grid search. This two-stage approach allows us to hone in on the computationally...
