Numerical optimization

The aim is to find the extreme values (for example, maxima or minima) of a function f(x) or of an implicit equation g(x) = 0. In focus it is therefore the optimization problem max h(x). Or in other words, we search for a value that holds:

• (global minima)
• (global maxima)

Basically, two kinds of approaches exist to solve a complex optimization problem, as already mentioned:

• The pure deterministic approach
• The stochastic approach

Deterministic means in this chapter to follow strict rules to achieve the maxima without any randomness included. While the numerical deterministic solution of the problem depends on the analytical properties of the objective function h (for example, convexity and smoothness), the stochastic approach is of more general use.

For the following examples we use the following function, where afterwards we want to find its minimum. The optima of our modified 2D Rosenbrock function (mountains) should be at (1,1):

mountains <- function(v) { 
  (1 ...