Introduction

SVMs are a method of binary classification. The basic idea is to find a linear separating line in two dimensions (or hyperplane for more dimensions) between the two classes. We first assume that the binary class targets are -1 or 1, instead of the prior 0 or 1 targets. Since there may be many lines that separate two classes, we define the best linear separator that maximizes the distance between both classes:

Figure 1

Given two separable classes, o and x, we wish to find the equation for the linear separator between the two. The left-hand graph shows that there are many lines that separate the two classes. The right-hand graph shows the unique maximum margin line. The margin width is given by . This line is found by minimizing the L2 norm of A.

We can write such a hyperplane as follows:

Here, A is a vector of our partial slopes and x is a vector of inputs. The width...