Kernels for SVM

With a kernel trick, a 2D space is converted into a 3D space using a mapping function such that the nonlinear data can be classified or separated in a higher dimension (see Figure 5.4). The transformation of original data for mapping into the new space is done via kernel. The kernel function defines inner products (measure of similarity) in the transformed space.

The compute and storage requirements of SVMs increase with the number of training examples. The core of the algorithm is a quadratic programming problem separating support vectors from the training dataset. A linear kernel, which is just a dot product, is the fastest implementation of SVM. A few examples of linear and nonlinear kernels are shown in Figure 5.5a. The most common nonlinear SVM kernels are radial basis function (RBF), sigmoid, and polynomial.

Figure 5.4: (a) Example of non-linear separator (L), and (b) Data effectively classified in higher dimension

SVMs are very...