Tuning hyperparameters
Many models already mentioned in this chapter have multiple parameters that determine how the model will perform. Selecting some is relatively straightforward, but there are many that we simply cannot set intuitively. That's where hyperparameters-tuning comes to play. The hyperparameters-tuning methods help us select the best (or close to) set of parameters that maximizes some metric we defined.
In this recipe, we will show you two approaches for hyperparameter-tuning.
Getting ready
To execute this recipe, you will need a working Spark environment and you would have already loaded the data into the forest DataFrame. You would also have gone through all the previous recipes as we assume you have a working knowledge of Transformers, Estimators, Pipelines, and some of the regression models.
No other prerequisites are required.
How to do it...
We start with grid search. It is a brute-force method that simply loops through specific values of parameters, building new models and...
