We will now move on to Rainbow networks, which is a confluence of several different DQN improvements. Since the original DQN paper, several different improvements were proposed with notable success. This motivated DeepMind to combine several different improvements into an integrated agent, which they refer to as the Rainbow DQN. Specifically, six different DQN improvements are combined into one integrated Rainbow DQN agent. These six improvements are summarized as follows:
- DDQN
- Dueling network architecture
- Prioritized experience replay
- Multi-step learning
- Distributional RL
- Noisy nets
