Roboschool
To experiment with the methods in this chapter, we'll use roboschool, which uses PyBullet as a physics engine and has 13 environments of various complexity. PyBullet has similar environments, but at the time of writing it wasn't possible to create several instances of the same environment due to internal OpenGL issue. In this chapter, we'll get in touch with two problems: RoboschoolHalfCheetah-v1, which models a two-legged creature and RoboschoolAnt-v1, which has four legs. The state and action spaces of them are very similar to the Minitaur environment that we saw in the previous chapter: the state includes characteristics from joints and actions are activations of those joints. The goal for both is to move as far as possible, minimizing the energy spent.
To install roboschool, you need to follow the instructions on https://github.com/openai/roboschool. This requires...