Unveiling Rainbow DQN

The author of Rainbow: Combining Improvements in Deep Reinforcement Learning, Matteo Hessel (https://arxiv.org/search/cs?searchtype=author&query=Hessel%2C+M), did several comparisons against other state-of-the-art models in DRL, many of which we have already looked at. They performed these comparisons against the standard 2D classic Atari games with impressive results. Rainbow DQN outperformed all of the current state-of-the-art algorithms. In the paper, they used the familiar classic Atari environment. This is fine since DeepMind has a lot of data for that environment that is specific to applicable models to compare with. However, many have observed that the paper lacks a comparison between PG methods, such as PPO. Of course, PPO is an OpenAI advancement and it may have been perceived by Google DeepMind to be an infringement or just wanting to avoid...