Building blocks for distributed Deep Reinforcement Learning for accelerated training
The previous recipes in this chapter discussed how you could scale your Deep RL training using TensorFlow 2.x’s distributed execution APIs. While it was straightforward after understanding the concepts and the implementation style, training Deep RL agents with more advanced architectures such as Impala and R2D2 requires RL building blocks such as distributed parameter servers and distributed experience replay. This chapter will walk through the implementation of such building blocks for distributed RL training. We will be using the Ray distributed computing framework to implement our building blocks.
Let’s get started!
Getting ready
To complete this recipe, you will first need to activate the tf2rl-cookbook
Python/conda virtual environment. Make sure to update the environment to match the latest conda environment specification file (tfrl-cookbook.yml
) in the cookbook’s...