Installing OpenAI is not a straightforward task; there are a set of steps that have to be correctly followed for setting the system up and running it. Now, let's see how to set up our machine and install OpenAI Gym and Universe.

All the examples in the book use the Anaconda version of Python. Anaconda is an open source distribution of Python. It is widely used for scientific computing and processing a large volume of data. It provides an excellent package management environment. It provides support for Windows, macOS, and Linux. Anaconda comes with Python installed along with popular packages used for scientific computing such as NumPy, SciPy, and so on.

To download Anaconda...

With OpenAI Gym, we can simulate a variety of environments and develop, evaluate, and compare RL algorithms. Let's now understand how to use Gym.

Let's see how to simulate a basic cart pole environment:

1. First, let's import the library:

import gym

2. The next step is to create a simulation instance using the make function:

env = gym.make('CartPole-v0')

3. Then we should initialize the environment using the reset method:

env.reset()

4. Then we can loop for some time steps and render the environment at each step:

for _ in range(1000):
    env.render()
    env.step(env.action_space.sample())

The complete code is as follows:

import gym 
env = gym.make('CartPole-v0')
env...

OpenAI Universe provides a wide range of realistic gaming environments. It is an extension to OpenAI Gym. It provides the ability to train and evaluate agents on a wide range of simple to real-time complex environments. It has unlimited access to many gaming environments.

Let's learn how to build a video game bot which plays a car racing game. Our objective is that the car has to move forward without getting stuck on any obstacles or hitting other cars.

First, we import the necessary libraries:

import gym
import universe # register universe environment
import random

Then we simulate our car racing environment using the make function:

env = gym.make('flashgames.NeonRace-v0...

TensorFlow is an open source software library from Google which is extensively used for numerical computation. It is widely used for building deep learning models and is a subset of machine learning. It uses data flow graphs that can be shared and executed on many different platforms. Tensor is nothing but a multi-dimensional array, so when we say TensorFlow, it is literally a flow of multi-dimensional arrays (tensors) in the computation graph.

With Anaconda installed, installing TensorFlow becomes very simple. Irrespective of the platform you are using, you can easily install TensorFlow by typing the following command:

source activate universe
conda install -c conda-forge tensorflow

We can check whether the TensorFlow installation was successful by simply running the following Hello...

In this chapter, we learned how to set up our machine by installing Anaconda, Docker, OpenAI Gym, Universe, and TensorFlow. We also learned how to create simulations using OpenAI and how to train agents to learn in an OpenAI environment. Then we came across the fundamentals of TensorFlow followed by visualizing graphs in TensorBoard.

In the next chapter, Chapter 3, The Markov Decision Process and Dynamic Programming we will learn about Markov Decision Process and dynamic programming and how to solve frozen lake problem using value and policy iteration.

The question list is as follows:

1. Why and how do we create a new environment in Anaconda?
2. What is the need for using Docker?
3. How do we simulate an environment in OpenAI Gym?
4. How do we check all available environments in OpenAI Gym?
5. Are OpenAI Gym and Universe the same? If not, what is the reason?
6. How are TensorFlow variables and placeholders different from each other?
7. What is a computational graph?
8. Why do we need sessions in TensorFlow?
9. What is the purpose of TensorBoard and how do we start it?

You can further refer to these papers:

• OpenAI blog: https://blog.openai.com
• OpenAI environments: https://gym.openai.com/envs/
• TensorFlow official website: https://www.tensorflow.org/