The role of inner products in common learning algorithms
In Chapter 3, we introduced the Principal Component Analysis (PCA) unsupervised learning algorithm and showed how all the calculations in PCA could be expressed in terms of inner products between the feature vectors of the different points in the training data.
In Chapter 3, we also explained that the inner product 
between vectors 
and 
gives a measure of how similar vectors and are to each other. Since many learning algorithms are based on the idea that similar datapoints behave similarly, it is not surprising that PCA and indeed many other learning algorithms make use of inner products.
Like PCA, many classical statistical and machine learning algorithms can be expressed solely in terms of inner products, such as Linear Discriminant Analysis (LDA), Fisher Discriminant Analysis (FDA), Canonical Correlation Analysis (CCA), and Support Vector Machines (SVMs). We will refer to these types of algorithms as inner-product...
