How SVMs work
As we mentioned, SVMs can be used for classification and regression, and the implementation differs for each. We'll start with classification.
SVMs for classification
Let's start with a simple example of a dataset for classification, where we have two features on the two axes, and the target is represented by the color and shape. We want to separate the two types of datapoints with different shapes and colors, and we can see how several classification boundary lines could be drawn to separate them. We've currently drawn one potential separation line.
Figure 16.1: A two-class classification example with a hyperplane separating the two classes
SVMs allow us to mathematically find the best separation between groups of data. In the case of two dimensions, this is a line, but with higher dimensions, as we often see, this becomes a hyperplane. The objective of an SVM classifier is to find the optimal hyperplane to separate classes. This...