Tuning decision tree hyperparameters

Hyperparameters are not the same as parameters.

In machine learning, parameters are adjusted when the model is being tuned. The weights in linear and Logistic Regression, for example, are parameters adjusted during the build phase to minimize errors. Hyperparameters, by contrast, are chosen in advance of the build phase. If no hyperparameters are selected, default values are used.

Decision Tree regressor

The best way to learn about hyperparameters is through experimentation. Although there are theories behind the range of hyperparameters chosen, results trump theory. Different datasets see improvements with different hyperparameter values.

Before selecting hyperparameters, let's start by finding a baseline score using a DecisionTreeRegressor and cross_val_score with the following steps:

1. Download the 'bike_rentals_cleaned' dataset and split it into X_bikes (predictor columns) and y_bikes (training columns):

   df_bikes...