Forecasting with sequence-to-sequence models and attention
Let’s pick up the thread from Chapter 13, Common Modeling Patterns for Time Series, where we used Seq2Seq models to forecast a sample household (if you have not read the previous chapter, I strongly suggest you do it now) and modify the Seq2SeqModel
class to also include an attention mechanism.
Notebook alert:
To follow along with the complete code, use the notebook named 01-Seq2Seq_RNN_with_Attention.ipynb
in the Chapter14
folder and the code in the src
folder.
We are still going to inherit the BaseModel
class we have defined in src/dl/models.py
, and the overall structure is going to be very similar to the Seq2SeqModel
class. The key difference will be that in our new model, with attention, we do not accept a fully connected layer as the decoder. It is not because it is not possible, but for convenience and brevity of the implementation. In fact, implementing a Seq2Seq model with a fully connected...