What Are Hyperparameters?

Hyperparameters can be thought of as a set of dials and switches for each estimator that change how the estimator works to explain relationships in the data.

Have a look at Figure 8.1:

Figure 8.1: How hyperparameters work

If you read from left to right in the preceding figure, you can see that during the tuning process we change the value of the hyperparameter, which results in a change to the estimator. This in turn causes a change in model performance. Our objective is to find hyperparameterization that leads to the best model performance. This will be the optimal hyperparameterization.

Estimators can have hyperparameters of varying quantities and types, which means that sometimes you can be faced with a very large number of possible hyperparameterizations to choose for an estimator.

For instance, scikit-learn's implementation of the SVM classifier (sklearn.svm.SVC), which you will be introduced to later in the chapter...