Recurrent Neural Networks

In the previous chapter, we marveled over the visual cortex and leveraged some insights from the way it processes visual signals to inform the architecture of Convolutional Neural Networks (CNNs), which form the base of many state-of-the-art computer vision systems. However, we do not understand the world around us with vision alone. Sound, for one, also plays a very important role. More specifically, we humans love to communicate and express intricate thoughts and ideas through sequences of symbolic reductions and abstract representations. Our built-in hardware allows us to interpret vocalizations or demarcations thereof, composing the base of human thought and collective understandings, upon which more complex representations (such as human languages, for instance) may be composed. In essence, these sequences of symbols are reduced representations of...