Black-box methods

To begin with, let’s discuss the whole family of black-box methods and how it differs from what we’ve covered so far. Black-box optimization methods are the general approach to the optimization problem, when you treat the objective that you’re optimizing as a black box, without any assumptions about the differentiability, the value function, the smoothness of the objective, and so on. The only requirement that those methods expose is the ability to calculate the fitness function, which should give us the measure of suitability of a particular instance of the optimized entity at hand. One of the simplest examples in this family is random search, which is when you randomly sample the thing you’re looking for (in the case of RL, it’s the policy, π(a|s)), check the fitness of this candidate, and if the result is good enough (according to some reward criteria), then you’re done. Otherwise, you repeat the process again...