Classifying data with support vector machines

Support Vector Machines (SVM) are another group of classification and regression models that have proven to be quite powerful in many situations. The intuition behind them is quite straightforward to understand, but we'll see that their power comes mostly from a mathematical technique that's used in many other ML algorithms, called the kernel trick.

Intuition

Let's consider a simple classification problem where we want to classify samples into two categories. The following is a graph containing some randomly generated data for this problem:

Figure 12.5 – Simple classification problem data

Intuitively, with such data, finding a straight line to cleanly separate the two categories seems simple. However, we quickly see that there are a lot of different solutions, as shown in the following graph:

Figure 12.6 – Three possible linear classifiers

So, how...