Mathematical optimization

Optimization algorithms try to find the optimal solution for a problem, for instance finding the maximum or the minimum of a function. The function can be linear or non-linear. The solution could also have special constraints. For example, the solution may not be allowed to have negative values. Several optimization algorithms are provided by the scipy.optimize module. One of the algorithms is a least squares fitting function, leastsq. When calling this function, we are required to provide a residuals (error terms) function. This function is used to minimize the sum of the squares of the residuals. It corresponds to our mathematical model for the solution. Also, it is necessary to give the algorithm a starting point. This should be a best guess—as close as possible to the real solution. Otherwise, execution will stop after about 800 iterations.