Solving optimization problems
Those familiar with solving machine learning or optimization problems will know that there are algorithms that are frequently used to find the global optimum (minimum or maximum) value of a function. The function is typically called the objective function and has many variables and produces a multidimensional landscape, such as a mountain range, with high and low elevations.
Figure 5.1 – Local minimum versus global minimum
Typical methods used to find the global optimum include gradient descent, simulated annealing, or various genetic algorithms. Even though these are successful and fast algorithms, there is a chance that while searching a large landscape produced by a function with many variables, these algorithms often settle on a minimum (or maximum) that is not the global optimum value but a local optimum. These algorithms use random starting points and probabilistic behavior in moving around a landscape, with the intention...
