Introduction to this book

This book offers a practical guide for those eager to learn about reinforcement learning. We will take a hands-on approach toward learning about reinforcement learning by going through numerous examples of algorithms and their applications. Each chapter focuses on a particular use case and introduces reinforcement learning algorithms that are used to solve the given problem. Some of these use cases rely on state-of-the-art algorithms; hence through this book, we will learn about and implement some of the best-performing algorithms and techniques in the industry.

The projects increase in difficulty/complexity as you go through the book. The following table describes what you will learn from each chapter:

Chapter name	The use case/problem	Concepts/algorithms/technologies discussed and used
Balancing Cart Pole	Control horizontal movement of a cart to balance a vertical bar	OpenAI Gym framework, Q-Learning
Playing Atari Games	Play various Atari games at human-level proficiency	Deep Q-Networks
Simulating Control Tasks	Control agents in a continuous action space as opposed to a discrete one	Deterministic policy gradients (DPG), Trust Region Policy Optimization (TRPO), multi-tasking
Building Virtual Worlds in Minecraft	Navigate a character in the virtual world of Minecraft	Asynchronous Advantage Actor-Critic (A3C)
Learning to Play Go	Go, one of the oldest and most complex board games in the world	Monte Carlo tree search, policy and value networks
Creating a Chatbot	Generating natural language in a conversational setting	Policy gradient methods, Long Short-Term Memory (LSTM)
Auto Generating a Deep Learning Image Classifier	Create an agent that generates neural networks to solve a given task	Recurrent neural networks, policy gradient methods (REINFORCE)
Predicting Future Stock Prices	Predict stock prices and make buy and sell decisions	Actor-Critic methods, time-series analysis, experience replay

Expectations

This book is best suited for the reader who:

• Has intermediate proficiency in Python
• Possesses a basic understanding of machine learning and deep learning, especially for the following topics:
  • Neural networks
  • Backpropagation
  • Convolution
  • Techniques for better generalization and reduced overfitting
• Enjoys a hands-on, practical approach toward learning

Since this book serves as a practical introduction to the field, we try to keep theoretical content to a minimum. However, it is advisable for the reader to have basic knowledge of some of the fundamental mathematical and statistical concepts on which the field of machine learning depends. These include the following:

• Calculus (single and multivariate)
• Linear algebra
• Probability theory
• Graph theory

Having some experience with these subjects would greatly assist the reader in understanding the concepts and algorithms we will cover throughout this book.

Hardware and software requirements

The ensuing chapters will require you to implement various reinforcement learning algorithms. Hence a proper development environment is necessary for a smooth learning journey. In particular, you should have the following:

• A computer running either macOS or the Linux operating system (for those on Windows, try setting up a Virtual Machine with a Linux image)
• A stable internet connection
• A GPU (preferably)

We will exclusively use the Python programming language to implement our reinforcement learning and deep learning algorithms. Moreover, we will be using Python 3.6. A list of libraries we will be using can be found on the official GitHub repository, located at (https://github.com/PacktPublishing/Python-Reinforcement-Learning-Projects). You will also find the implementations of every algorithm we will cover in this book.

Installing packages

Assuming you have a working Python installation, you can install all the required packages using the requirements.txt file found in our repository. We also recommend you create a virtualenv to isolate your development environment from your main OS system. The following steps will help you construct an environment and install the packages:

# Install virtualenv using pip
$ pip install virtualenv

# Create a virtualenv
$ virtualenv rl_projects

# Activate virtualenv
$ source rl_projects/bin/activate

# cd into the directory with our requirements.txt
(rl_projects) $ cd /path/to/requirements.txt

# pip install the required packages
(rl_projects) $ pip install -r requirements.txt

And now you are all set and ready to start! The next few sections of this chapter will introduce the field of reinforcement learning and will also provide a refresher on deep learning.