In the previous recipe, we treated each observed image in the Pong environment as a grayscale array and fed it to a fully connected neural network. Flattening an image may actually result in information loss. Why don’t we use the image as input instead? In this recipe, we will incorporate convolutional neural networks (CNNs) into the DQN model.
A CNN is one of the best neural network architectures to deal with image inputs. In a CNN, the convolutional layers are able to effectively extract features from images, which will be passed on to downstream, fully connected layers. An example of a CNN with two convolutional layers is depicted here:
As you can imagine, if we simply flatten an image into a vector, we will lose some information on where the ball is located, and where the two players are. Such information is significant...
