Exploring the evolution of recurrent networks
Recurrent networks have been around since the 80s. In this section, we will explore the evolution of the recurrent network architecture since its inception. We will discuss and reason about the developments that were made to the architecture by going through the key milestones in the evolution of RNNs. Before jumping right into the timelines, we’ll quickly review the different types of RNNs and how they relate to a general feed-forward neural network.
Types of recurrent neural networks
While most supervised machine learning models model one-to-one relationships, RNNs can model the following types of input-output relationships:
- Many-to-many (instantaneous)
Example: Named entity recognition – Given a sentence/text, tag the words with named entity categories such as names, organizations, locations, and so on.
- Many-to-many (encoder-decoder)
Example: Machine translation (say, from...
