Performance considerations

You may have already observed that the training of a model for the support vector regression on a large dataset is time consuming. The performance of the support vector machine depends on the type of optimizer (for example, a sequential minimal optimization) selected to maximize the margin during training:

• A linear model (a SVM without kernel) has an asymptotic time complexity O(N) for training N labeled observations.

• Nonlinear models rely on kernel methods formulated as a quadratic programming problem with an asymptotic time complexity of O(N³)

• An algorithm that uses sequential minimal optimization techniques, such as index caching or elimination of null values (as in LIBSVM), has an asymptotic time complexity of O(N²) with the worst case scenario (quadratic optimization) of O(N³)

• Sparse problems for very large training sets (N > 10,000) also have an asymptotic time of O(N²)

The time and space complexity of the kernelized support vector machine has been receiving...