Simple RNN architecture does not always work due to the problem of exploding and vanishing gradients, hence improved RNN architectures are used such as LSTM networks. TensorFlow provides API to create LSTM RNN architectures.
In the example showcased in the previous section, to change the Simple RNN to the LSTM network, all we have to do is change the cell type as follows:
cell = tf.nn.rnn_cell.LSTMCell(state_size)
The rest of the code remains the same as TensorFlow does the work of creating the gates inside the LSTM cell for you.
The complete code for the LSTM model is provided in the notebook ch-07a_RNN_TimeSeries_TensorFlow.
However, with LSTM, we had to run the code for 600 epochs in order to get results closer to a basic RNN. The reason being that LSTM has more parameters to learn, hence it needs more training iterations. For our simple example, it seems...
