RNN architectures and applications

RNNs can be categorized into many-to-one, one-to-many, many-to-many (synced), and many-to-many (unsynced) based on their input and output. From the perspective of hidden layers, the most commonly used RNN architectures include the basic, vanilla RNNs, and the bidirectional ones, LSTM and GRU. We will focus on these four architectures of RNNs, and will start by briefly mentioning those four categories by input and output.

Architectures by input and output

Many-to-one: The many-to-one architecture is probably the most intuitive one. We can input as many elements or time steps in the sequence, but the model produces only one output after going through the entire sequence. Its...