Improving your RNN model

When working on a problem using RNN (or any other network), your process looks like this:

First, you come up with an idea for the model, its hyperparameters, the number of layers, how deep the network should be, and so on. Then the model is implemented and trained in order to produce some results. Finally, these results are assessed and the necessary modifications are made. It is rarely the case that you'll receive meaningful results from the first run. This cycle may occur multiple times until you are satisfied with the outcome. 

Considering this approach, one important question comes to mind: How can we change the model so the next cycle produces better results?

This question is tightly connected to your understanding of the network's results. Let's discuss that now. 

As you already know, in the beginning of each...