Understanding the importance of the simulation in reinforcement learning
As we mentioned multiple times so far, and especially in the first chapter when we talked about RL success stories, RL's hunger for data is orders of magnitude greater than that of deep supervised learning. That is why it takes many months to train some complex RL agents, over millions and billions of iterations. Since it is often impractical to collect such data in a physical environment, we heavily rely on simulation models in training RL agents. This brings some challenges along with it:
- Many businesses don't have a simulation model of their processes. This makes it challenging to bring the RL technology to the use of such companies.
- When a simulation model exists, it is often too simplistic to capture the real-world dynamics. As a result, RL models could easily overfit to the simulation environment and may fail in deployment. It takes significant time and resources to calibrate and validate...
