In the first recipe, we will start by getting familiar with the Cliff Walking environment, which we will solve with TD methods in upcoming recipes.
Cliff Walking is a typical gym environment, with long episodes without a guarantee of termination. It is a grid problem with a 4 * 12 board. An agent makes a move up, right, down, and left at a step. The bottom-left tile is the starting point for the agent, and the bottom-right is the winning point where an episode will end if it is reached. The remaining tiles in the last row are cliffs where the agent will be reset to the starting position after stepping on any of them, but the episode continues. Each step the agent takes incurs a -1 reward, with the exception of stepping on the cliffs, where a -100 reward is incurred.
