Environments
Most of the environments that include continuous action spaces are related to the physical world, so physics simulations are normally used. There are lots of software packages that can simulate physical processes, from very simple, open-source tools to complex, commercial packages that can simulate multiphysics processes (such as fluid, burning, and strength simulations). In the case of robotics, one of the most popular packages is MuJoCo, which stands for Multi-Joint Dynamics with Contact (www.mujoco.org). This is a physics engine in which you can define the components of the system, their interaction and properties. Then the simulator is responsible for solving the system by taking into account your intervention and finding the parameters (usually the location, velocities, and accelerations) of the components. This makes it ideal as a playground for RL environments, as you can define fairly complicated systems (such as multipede robots or robotic arms or humanoids) and ...