4.9.2 Solving a least square problem with SVD

A linear equation system , with  being an  matrix and , is called an overdetermined linear system. In general, it has no classical solution and you seek a vector  with the property:

Here,  denotes the Euclidean vector norm .

This problem is called a least square problem. A stable method to solve it is based on factorizing , with  being an  orthogonal matrix,  an  orthogonal matrix, and  an matrix with the property  for all . This factorization is called a singular value decomposition (SVD).

We write

with a diagonal matrix . If we assume that  has full rank, then  is invertible and it can be shown that 

holds. 

If we split with being an  submatrix, then the preceding equation can be simplified to:

 

SciPy provides a function called svd, which we use to solve this task:

import scipy.linalg...