We have come to a point in this book where we have learned about all the major concepts of DRL. There will be more tools we will throw at you in later chapters, such as the one we showed in this section, but if you have made it this far, you should consider yourself knowledgeable of DRL. As such, consider building your own tools or enhancements to DRL, not unlike the one we'll show in this section. If you are wondering if you need to have the math worked out first, then the answer is no. It can often be more intuitive to build these models in code first and then understand the math later.
Actor-critic with experience replay (ACER) provides another advantage by adjusting sampling based on past experiences. This concept was originally introduced by DeepMind in a paper titled Sample Efficient Actor-Critic with Experience Replay and...
