What do you get with Print?

Instant access to your digital eBook copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

AI Assistant (beta) to help accelerate your learning

Markov Decision Processes and Dynamic Programming

In this chapter, we will continue our practical reinforcement learning journey with PyTorch by looking at Markov decision processes (MDPs) and dynamic programming. This chapter will start with the creation of a Markov chain and an MDP, which is the core of most reinforcement learning algorithms. You will also become more familiar with Bellman equations by practicing policy evaluation. We will then move on and apply two approaches to solving an MDP: value iteration and policy iteration. We will use the FrozenLake environment as an example. At the end of the chapter, we will demonstrate how to solve the interesting coin-flipping gamble problem with dynamic programming step by step.

The following recipes will be covered in this chapter:

Creating a Markov chain
Creating an MDP
Performing policy evaluation
Simulating the FrozenLake...

Key benefits

Use PyTorch 1.x to design and build self-learning artificial intelligence (AI) models

Implement RL algorithms to solve control and optimization challenges faced by data scientists today

Apply modern RL libraries to simulate a controlled environment for your projects

Description

Reinforcement learning (RL) is a branch of machine learning that has gained popularity in recent times. It allows you to train AI models that learn from their own actions and optimize their behavior. PyTorch has also emerged as the preferred tool for training RL models because of its efficiency and ease of use. With this book, you'll explore the important RL concepts and the implementation of algorithms in PyTorch 1.x. The recipes in the book, along with real-world examples, will help you master various RL techniques, such as dynamic programming, Monte Carlo simulations, temporal difference, and Q-learning. You'll also gain insights into industry-specific applications of these techniques. Later chapters will guide you through solving problems such as the multi-armed bandit problem and the cartpole problem using the multi-armed bandit algorithm and function approximation. You'll also learn how to use Deep Q-Networks to complete Atari games, along with how to effectively implement policy gradients. Finally, you'll discover how RL techniques are applied to Blackjack, Gridworld environments, internet advertising, and the Flappy Bird game. By the end of this book, you'll have developed the skills you need to implement popular RL algorithms and use RL techniques to solve real-world problems.

What you will learn

Use Q-learning and the state–action–reward–state–action (SARSA) algorithm to solve various Gridworld problems

Develop a multi-armed bandit algorithm to optimize display advertising

Scale up learning and control processes using Deep Q-Networks

Simulate Markov Decision Processes, OpenAI Gym environments, and other common control problems

Select and build RL models, evaluate their performance, and optimize and deploy them

Use policy gradient methods to solve continuous RL problems

What do you get with Print?

Instant access to your digital eBook copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

AI Assistant (beta) to help accelerate your learning

Frequently bought together

€36.99

PyTorch 1.x Reinforcement Learning Cookbook

€32.99

Reinforcement Learning Algorithms with Python

€29.99

Total € 99.97

AustinSF Jan 24, 2020

Got this book with the recommendation from one of my friend, this book provides good guidance for the people who wish to dive into the career in machine learning, with easy to understand explanations and hands-on code examples. It also teaches you how to think and tackle challenges like an ML engineer using reinforcement learning, which I think it's more important for a newbie like me.

Amazon Verified review

kepler man Oct 28, 2020

DQNの数式をOpenAIのGymのゲームでPyTorchで組んで具現化するレシピ集です。理論的な事は別の専門書に委ねるとして、数式を実際に組んでみるとこの様にできるという数多くの例を段階的に教示してあり、実際に動くと楽しくなります。deepmind社もAtariのゲームで発表していますので、このレシピ集は教材として充分でしょう。サンプルを試す前にgymでインストールしておくのはatariの他にbox2dやclassic_controlが必要で、3DゲームのMuJoCo(お試しライセンス付き)のサンプルは載っていません。あと最後の方にpygameのインストールが必要なDQNも載っています。対応マシンは、Windows、MacOS、Linuxのi686,x86_64マシンで、Pythonの3.6又は3.7です。aarch64系のJetson NANOでも少し工夫しましたが、ほぼサンプルは動かせました。その時のメモ:「sudo apt-get install -y cmake zlib1g-dev libjpeg-dev xvfb ffmpeg xorg-dev libboost-all-dev libsdl2-dev swiggitからのgymのsetup.pyをi686,x86_64用のopencv-pythonを外して編集し、'atari':['atari_py==0.2.0','Pillow'] だけにします。代替えに sudo apt-get install python3-opencv で、旧い3系を入れ、pip3 install -e '.[atari, box2d, classic_control]'この時に、--userで.localに入れるとenv.render()をかけるとライブラリーの関係でエラーになるのでそのまま普通の場所にインストール。env設定でメモリーに一旦ロードされると誤記でもしてSyntaxErrorをわざと起こさない限りクリアーでリセットがし難いとかの裏技発見。gym 0.17.3が依存するpyglet<=1.5.0,>=1.4.0の1.5.0はバグ、範囲外だがpyglet-1.5.7の最新版の方が調子が良い。」

Matthew Emerick Jul 21, 2020

About This BookThis is a cookbook style (obviously) technical book that takes the intermediate machine learning developer into the world of PyTorch reinforcement leaning. It covers everything from setting up PyTorch and the OpenAI Gym environment to a final capstone project, the later of which I've never seen in a technical cookbook before.Who is This For?As per the preface, this book is written for a developer who has experience with machine learning. It doesn't teach you PyTorch, per se, but you can pick it up. Knowing PyTorch beforehand would probably let you get more out of this book, though.Why Was This Written?There is a massive market for books on the various aspects of artificial intelligence. Reinforcement learning is one of the areas that has seen a lot of interest lately, and rightfully so. I think that this is the kind of book that we need: it pulls in a specific and popular technology and looks at a single subfield of machine learning.OrganizationThis book sticks to a simple organization which makes it easier to find the section that you are looking for. The first chapter is all about setting up your system with PyTorch and the OpenAI Gym environment. Then it touches on a few simpler algorithms to introduce the reader to both technologies. The next seven chapters discuss reinforcement algorithms of increasing complexity. The book then finishes with a chapter with a capstone project, which I have never seen in a cookbook before. I think I like it, as it brings everything together.Did This Book Succeed?This book did everything that it tries to do. It checks all of the boxes on its todo list and gives the reader a lot of new knowledge of reinforcement learning and PyTorch. I agree with the supposition that the reader should have a good understanding of machine learning already, as the book doesn't go into much detail, nor should it; this is an intermediate book.Rating and Final ThoughtsI like this book and I think it's a good addition to anyone's collection. The capstone project is a nice touch. I just don't think it's exceptional. The author clearly has technical expertise, but writing is not their strongest suit. I hope the author keeps writing, though, as there is a lot to learn from him. All said and done, I give this book a 4 out of 5.

PyTorch 1.x Reinforcement Learning Cookbook: Over 60 recipes to design, develop, and deploy self-learning AI models using Python

What do you get with Print?

PyTorch 1.x Reinforcement Learning Cookbook

Markov Decision Processes and Dynamic Programming

Technical requirements

Creating a Markov chain

Creating an MDP

Performing policy evaluation

Simulating the FrozenLake environment

Solving an MDP with a value iteration algorithm

Solving an MDP with a policy iteration algorithm

Solving the coin-flipping gamble problem

Page 1 of 9

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Product Details

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

People who bought this also bought

About the author

FAQs

PyTorch 1.x Reinforcement Learning Cookbook: Over 60 recipes to design, develop, and deploy self-learning AI models using Python

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

People who bought this also bought

About the author

FAQs