Singular value decomposition

This method is based on a theorem that states that a matrix X d x N can be decomposed as follows:

Here:

• U is a d x d unitary matrix
• ∑ is a d x N diagonal matrix where the diagonal entries si are called singular values
• V is an N x N unitary matrix

In our case, X can be composed by the feature's vectors , where each is a column. We can reduce the number of dimensions of each feature vector d, approximating the singular value decomposition. In practice, we consider only the largest singular values so that:

t represents the dimension of the new reduced space where the feature vectors are projected. A vector x⁽ⁱ⁾ is transformed in the new space using the following formula:

This means that the matrix (not ) represents the feature vectors in the t dimensional space.

Note that it is possible to show that this method is very similar to the PCA; in fact, the scikit-learn library uses SVD to implement PCA.