In this chapter, we explored the concept of 3D worlds for not only games but the real world. The real world, and to a greater extent 3D worlds, are the next great frontier in DRL research. We looked at why 3D creates nuances for DRL that we haven't quite figured out how best to solve. Then, we looked at using 2D visual observation encoders but tuned for 3D spaces, with variations in the Nature CNN and ResNet or residual networks. After that, we looked at the Unity Obstacle Tower Challenge, which challenged developers to build an agent capable of solving the 3D multi-task environment.
From there, we looked at the winning entries use of Prierarchy; a form of HRL in order to manage multiple task spaces. We also looked at the code in detail to see how this reflected in the winners modified PPO implementation. Lastly, we finished the chapter by looking at Habitat; an advanced...
