We are now entering areas where we will start looking at state-of-the-art algorithms, at least at the time of writing. Of course, that will likely change and things will advance. For now, though, the proximal policy optimization algorithm (PPO), was introduced by OpenAI, is considered a state-of-the-art deep reinforcement learning algorithm. As such, the sky is the limit as to what environments we can throw at this problem. However, in order to quantify our progress and for a variety of other reasons, we will continue to baseline against the Lunar Lander environment.
The PPO algorithm is just an extension and simplification of the trust region policy optimization (TRPO) algorithm we covered in Chapter 8, Policy Gradient Methods, but with a few key differences. PPO is also much simpler to understand and follow. For these reasons, we will...
