Large-scale Deep RL agent training using Ray, Tune, and RLLib

In the previous recipe, we got a flavor of how to implement distributed RL agent training routines from scratch. Since most of the components used as building blocks have become a standard way of building Deep RL training infrastructure, we can leverage an existing library that maintains a high-quality implementation of such building blocks. Fortunately, with our choice of ray as the framework for distributed computing, we are in a good place. Tune and RLLib are two libraries built on top of ray, and are available together with Ray, that provide highly scalable hyperparameter tuning (Tune) and RL training (RLLib). This recipe will provide a curated set of steps to get you acquainted with ray, Tune, and RLLib so that you can utilize them to scale your Deep RL training routines. In addition to the recipe discussed here in the text, the cookbook’s code repository for this chapter will have a handful of additional recipes...