Support vector regression

scikit-learn provides a support vector regressor based on a very simple variant of the algorithm already described (see the original documentation for further information). The real power of this approach resides in the usage of non-linear kernels (in particular, polynomials); however, the user is advised to evaluate the degree progressively because the complexity can grow rapidly, together with the training time.

For our example, I've created a dummy dataset based on a second-order noisy function:

>>> nb_samples = 50 

>>> X = np.arange(-nb_samples, nb_samples, 1) 
>>> Y = np.zeros(shape=(2 * nb_samples,)) 

>>> for x in X: 
       Y[int(x)+nb_samples] = np.power(x*6, 2.0) / 1e4 + np.random.uniform(-2, 2)

The dataset in plotted in the following figure:

In order to avoid a very long training process, the model is evaluated with degree set to 2. The epsilon parameter allows us to specify a soft margin for predictions; if a predicted...