Kernels and the kernel trick

Recall that the perceptron separates instances of the positive class from instances of the negative class using a hyperplane as a decision boundary. The decision boundary is given by the following formula:

Predictions are made using the following function:

Note

Note that we previously indicated the inner product

with wTx. To be consistent with the notational conventions used for SVM, we will adopt the former in this chapter.

While the proof is beyond the scope of this chapter, we can write the model differently. The following expression of the model is called the dual form. The expression we used previously is the primal form.

The most important difference between the primal and dual forms is that the primal form computes the inner product of the model parameters and the test instance's feature vector, while the dual form computes the inner product of the training instance's and the test instance's feature vector. Shortly we will exploit this property of the dual form...