Until now, all of our experiments have been with the heuristics brain using an internal RL algorithm called Q-learning. Now that we realize that C# scripts will only go so far, we can look at using ML-Agent external brains developed with Python. The preferred method for Unity is to run a Jupyter Python notebook externally to control the Unity training environment. This requires us to build a special Unity environment. Go through the following steps to learn how to configure the 3D ball environment for external training:
- Open the Unity editor and load the ML-Agents demo unityenvironment project. If you still have it open from the last chapter, then that will work as well.
- Open the 3DBall.scene in the ML-Agents/Examples/3DBall folder.
- Locate the Brain3DBrain object in the Hierarchy window and select it. In the Inspector window set the Brain Type to External...