Least squares is a powerful technique for finding a function from a relatively small family of potential functions that best describe a particular set of data. This technique is especially common in statistics. For example, least squares is used in linear regression problems – here, the family of potential functions is the collection of all linear functions. Usually, this family of functions that we try to fit has relatively few parameters that can be adjusted to solve the problem.
The idea of least squares is relatively simple. For each data point, we compute the square of the residual – the difference between the value of the point and the expected value given a function – and try to make the sum of these squared residuals as small as possible (hence least squares).
In this recipe, we'll learn how to use least squares to fit a curve to a sample set of data.
Getting ready
For...
