Now that we've introduced black-box evolutionary algorithms and evolution strategies in particular, we are ready to put what we have just learned into practice. The paper called Evolution Strategies as a Scalable Alternative to Reinforcement Learning by OpenAI made a major contribution to the adoption of evolution strategies as an alternative to reinforcement learning algorithms.
The main contribution of this paper is in the approach that scales ES extremely well with a number of CPUs. In particular, the new approach uses a novel communication strategy across CPUs that involves only scalars, and so it is able to scale across thousands of parallel workers.
Generally, ES requires more experience and thus is less efficient than RL. However, by spreading the computation across so many workers (thanks to the adoption of this new strategy), the task...
