Polynomial regression

The previous analysis has been centered around the idea of obtaining a linear equation to represent a given dataset. However, many datasets derive from non-linear relationships. Fortunately, there are alternative mathematical models from which to choose.

The simplest nonlinear functions are polynomials: y = f(x) b₀ + b₁x + b₂x² + …+b_dx^d, where d is the degree of the polynomial and b₀, b₁, b₂, ..., b_m are the coefficients to be determined.

Of course, a linear function is simply a first-degree polynomial: y = b₀ + b₁x. We have already solved that problem (in the previous derivation, we called the coefficients m and b instead of b₁ and b₀). We used the method of least squares to derive the formulas for the coefficients:

Those formulas were derived from the normal equations:

The equations were obtained by minimizing the sum of squares:

We can apply the same least squares method to find the best-fitting polynomial of any degree d for a given dataset, provided that d is less than...