Tuning gradient boosting trees

We will examine the California housing dataset with gradient boosting trees. Our overall approach will be the same as before:

1. Focus on important parameters in the gradient boosting algorithm:
   • max_features
   • max_depth
   • min_samples_leaf
   • learning_rate
   • loss
2. Create a parameter distribution where the most important parameters are varied.
3. Perform a random grid search. If using an ensemble, keep the number of estimators low at first.
4. Use the best parameters from the previous step with many estimators.

Getting ready

Load the California housing dataset and split the loaded dataset into training and testing sets:

%matplotlib inline 

from __future__ import division #Load within Python 2.7 for regular division...