You're reading from Deep Reinforcement Learning Hands-On A practical and easy-to-follow guide to RL from Q-learning and DQNs to PPO and RLHF

Product type Paperback

Published in Nov 2024

Publisher Packt

ISBN-13 9781835882702

Length 716 pages

Edition 3rd Edition

Languages

Python

Tools

PyTorch

Concepts

Deep Reinforcement Learning

Author (1):

Maxim Lapan

View More author details

Table of Contents (29) Chapters

Preface

1. Part 1 Introduction to RL FREE CHAPTER

2. What Is Reinforcement Learning?

3. OpenAI Gym API and Gymnasium

4. Deep Learning with PyTorch

5. The Cross-Entropy Method

6. Part 2 Value-based methods

7. Tabular Learning and the Bellman Equation

8. Deep Q-Networks

9. Higher-Level RL Libraries

10. DQN Extensions

11. Ways to Speed Up RL

12. Stocks Trading Using RL

13. Part 3 Policy-based methods

14. Policy Gradients

15. Actor-Critic Method: A2C and A3C

16. The TextWorld Environment

17. Web Navigation

18. Part 4 Advanced RL

19. Continous Action Space

20. Trust Region Methods

21. Black-Box Optimizations in RL

22. Advanced Exploration

23. Reinforcement Learning with Human Feedback

24. AlphaGo Zero and MuZero

25. RL in Discrete Optimization

26. Multi-Agent RL

27. Bibliography

28. Index

MuZero

The successor of AlphaGo Zero (published in 2017) was a method called MuZero, described by Schrittwieser et al. from DeepMind in the paper Mastering Atari, Go, chess and shogi by planning with a learned model [Sch+20] published in 2020. In this method, the authors made an attempt to generalize the method by removing the requirement of the precise game model, but still keeping the method in the model-based family. As we saw in the description of Alpha Go Zero, the game model is heavily used during the training process: in the MCTS phase, we use the game model to obtain the available actions in the current state and the new state of the game after applying the action. In addition, the game model provides the final game outcome: whether we have won or lost the game.

At first glance, it looks almost impossible to get rid of the model from the training process, but MuZero not only demonstrated how it could be done, but has also beaten the previous AlphaGo Zero records in...

The rest of the chapter is locked

You're reading from Deep Reinforcement Learning Hands-On A practical and easy-to-follow guide to RL from Q-learning and DQNs to PPO and RLHF

Table of Contents (29) Chapters

MuZero

Authors (1)

Personalised recommendations for you

You're reading from Deep Reinforcement Learning Hands-On A practical and easy-to-follow guide to RL from Q-learning and DQNs to PPO and RLHF

Table of Contents (29) Chapters

MuZero

Unlock this book and the full library FREE for 7 days

Authors (1)

Personalised recommendations for you