You're reading from Mastering Reinforcement Learning with Python

Product type Book

Published in Dec 2020

Publisher Packt

ISBN-13 9781838644147

Pages 544 pages

Edition 1st Edition

Languages

Python

Concepts

Reinforcement Learning

Author (1):

Enes Bilgin

Table of Contents (24) Chapters

Preface

1. Section 1: Reinforcement Learning Foundations

2. Chapter 1: Introduction to Reinforcement Learning

3. Chapter 2: Multi-Armed Bandits

4. Chapter 3: Contextual Bandits

5. Chapter 4: Makings of a Markov Decision Process

6. Chapter 5: Solving the Reinforcement Learning Problem

7. Section 2: Deep Reinforcement Learning

8. Chapter 6: Deep Q-Learning at Scale

9. Chapter 7: Policy-Based Methods

10. Chapter 8: Model-Based Methods

11. Chapter 9: Multi-Agent Reinforcement Learning

12. Section 3: Advanced Topics in RL

13. Chapter 10: Introducing Machine Teaching

14. Chapter 11: Achieving Generalization and Overcoming Partial Observability

15. Chapter 12: Meta-Reinforcement Learning

16. Chapter 13: Exploring Advanced Topics

17. Section 4: Applications of RL

18. Chapter 14: Solving Robot Learning

19. Chapter 15: Supply Chain Management

20. Chapter 16: Personalization, Marketing, and Finance

21. Chapter 17: Smart City and Cybersecurity

22. Chapter 18: Challenges and Future Directions in Reinforcement Learning

23. Other Books You May Enjoy

Leave a review - let other readers know what you think

Chapter 3: Contextual Bandits

A more advanced version of the multi-armed bandit is the contextual bandit (CB) problem, where decisions are tailored to the context they are made in. In the previous chapter, we identified the best performing ad in an online advertising scenario. In doing so, we did not use any information about, for instance, the user persona, age, gender, location, previous visits etc., which would have increased the likelihood of a click. Contextual bandits allow us to leverage such information, which makes them play a central role in commercial personalization and recommendation applications.

Context is similar to a state in a multi-step reinforcement learning (RL) problem, with one key difference. In a multi-step RL problem, the action an agent takes affects the states it is likely to visit in the subsequent steps. For example, while playing tic-tac-toe, an agent's action in the current state changes the board configuration (state) in a particular way, which...