Introduction to hyperparameter tuning

The method used to choose the best estimators for a particular dataset or choosing the best values for all hyperparameters is called hyperparameter tuning. Hyperparameters are parameters that are not directly learned within estimators. Their value is decided by the modelers.

For example, in the RandomForestClassifier object, there are a lot of hyperparameters, such as n_estimators, max_depth, max_features, and min_samples_split. Modelers decide the values for these hyperparameters.

Exhaustive grid search

One of the most important and generally-used methods for performing hyperparameter tuning is called the exhaustive grid search. This is a brute-force approach because it tries all of...