LSTM model

We have seen that RNNs have a memory that uses persistent previous information to be used in the current neural network processing. The previous information is used in the present task. However, the memory is short-term and we do not have a list of all of the previous information available for the neural node.

When we introduce a long-term memory into the RNN, we are able to remember a lot of previous information and use it for the current processing. This concept is called LSTM model of RNN, which has numerous use cases in video, audio, text prediction, and various other applications.

LSTMs were introduced by Hochreiter & Schmidhuber in 1997.

The LSTM network is trained using BPTT and diminishes the vanishing gradient problem. LSTMs have powerful applications in time series predictions and can create large, recurrent networks to address difficult sequence problems in machine learning.

LSTM have gates that make the long/short term memory possible. These are contained in memory...