Higher-Level RL Libraries
In Chapter 6, we implemented the deep Q-network (DQN) model published by DeepMind in 2015 [Mni+15]. This paper had a significant effect on the RL field by demonstrating that, despite common belief, it’s possible to use nonlinear approximators in RL. This proof of concept stimulated great interest in the deep Q-learning field and in deep RL in general.
In this chapter, we will take another step toward a practical RL by discussing higher-level RL libraries, which will allow you to build your code from higher-level blocks and focus on the details of the method that you are implementing, avoiding reimplementing the same logic multiple times. Most of the chapter will describe the PyTorch AgentNet (PTAN) library, which will be used in the rest of the book to prevent code repetition, so will be covered in detail.
We will cover:
-
The motivation for using high-level libraries, rather than reimplementing everything...
