Implementing policy gradients

In reinforcement learning, we cannot the error in our network directly, because we don't have a truth set for each step. We only receive feedback now and then. This is why we need the policy gradient to propagate the rewards back to the network. The rules to determine the best are called policies. The network for learning these policies is called policy network. This can be type of network, for example, a simple, two-layer FNN or a CNN. The more complex the environment, the more you will benefit from a complex network. When using a policy gradient, we draw an action of the output distribution of our policy network. Because the reward is not always directly available, we treat the action as correct. Later we use the discounted reward as a scalar and backpropagate this to the network weights.

In the following recipe, we will teach an to play Pong from OpenAI by implementing a policy gradient in TensorFlow. Pong is a great game to start with, because it is simple...