Exploring the evolution of recurrent networks
Recurrent networks have been around since the 80s. In this section, we will explore the evolution of the recurrent network architecture since its inception. We will discuss and reason about the developments that were made to the architecture by going through the key milestones in the evolution of (RNNs). Before jumping right into the timelines, we'll quickly review the different types of RNNs and how they relate to a general feed-forward neural network.
Types of recurrent neural networks
While most supervised machine learning models model one-to-one relationships, (RNNs) can model the following types of input-output relationships:
- Many-to-many (instantaneous)
Example: Named-entity-recognition: Given a sentence/text, tag the words with named entity categories such as names, organizations, locations, and so on.
- Many-to-many (encoder-decoder)
Example: Machine translation (say, from English text to German text): Takes in...
