MountainCar experiments
In this section, we will try to implement and compare the effectiveness of different exploration approaches on a simple, but still challenging, environment, which could be classified as a “classical RL” problem that is very similar to the familiar CartPole problem. But in contrast to CartPole, the MountainCar problem is quite challenging from an exploration point of view.
The problem’s illustration is shown in Figure 18.3 and it consists of a small car starting from the bottom of the valley. The car can move left and right, and the goal is to reach the top of the mountain on the right.
The trick here is in the environment’s dynamics and the action space. To reach the top, the actions need to be applied in a particular way to swing the car back and forth to speed it up. In other words, the agent needs to apply the actions for several...
