Singular value decomposition

Singular Value Decomposition (SVD) is probably the most famous decomposition you can do for linear operators and matrices. It is at the core of search engines and machine learning algorithms. Additionally, it can be used on any type of matrix, even rectangular ones. However, we will only look at square matrices.

Succinctly stated, it guarantees that for any matrix A, it can be decomposed into three matrices:

Whereas U is a unitary matrix, Σ (sigma) is a diagonal matrix with what is known as the singular values of A on its diagonal, and V is also a unitary matrix. It should be noted that this decomposition is not unique, and different matrices can be used for U, Σ, and V.

Let's look at an example. We have the following matrix A:

Without going through the math, I'm going to tell you that SVD can be used to get this decomposition:

Let's make sure...