Least squares approximation

Numerically, it is relatively simple to state the approximation problem for the least squares norm. This is the topic of this section.

Linear least squares approximation

In the context of linear least squares approximation, it is always possible to reduce the problem to solving a system of linear equations, as the following example shows:

Consider the sine function f(x) = sin(x) in the interval from 0 to 1. We choose as approximants the polynomials of second degree: {a₀ + a₁x + a₂x²}. To compute the values [a₀, a₁, a₂] that minimize this problem, we first form a 3 × 3 matrix containing the pairwise dot products (the integral of the product of two functions) of the basic functions {1, x, x²} in the given interval. Because of the nature of this problem, we obtain a Hilbert matrix of order 3:

[   < 1, 1 >    < 1, x >    < 1, x^2 > ]     [  1   1/2  1/3 ]
[   < x, 1 >    < x, x >    < x, x^2 > ]  =  [ 1/2  1/3  1/4 ]
[ < x...