The standard environment for building continuous control agents is the Mujoco environment. Mujoco stands for Multi-Joint dynamics with Contract and it is a full physics environment for training robotic or simulation agents. This environment provides a number of simulations that challenge some form of robotic control agent to perform a task, such as walking, crawling, and implementing several other physics control-based tasks. An example of the diversity of these environments is summarized well in the following image, which has been extracted from the Mujoco home page:
Obviously, we will want to use this cool environment. However, this package is not free and requires a license, but note that a 30-day trial is provided. Now for the bad news. The package is extremely difficult...
