Forecasting with sequence-to-sequence models and attention
Let’s pick up the thread from Chapter 13, Common Modeling Patterns for Time Series, where we used Seq2Seq models to forecast a sample household (if you have not read the previous chapter, I strongly suggest you do it now) and modify the Seq2SeqModel
class to also include an attention mechanism.
Notebook alert
To follow along with the complete code, use the notebook named 01-Seq2Seq RNN with Attention.ipynb
in the Chapter14
folder and the code in the src
folder.
We are still going to inherit the BaseModel
class we have defined in src/dl/models.py
, and the overall structure is going to be very similar to the Seq2SeqModel
class. The key difference will be that in our new model, with attention, we do not accept a fully connected layer as the decoder. It is not because it is not possible, but for convenience and brevity of the implementation. In fact, implementing a Seq2Seq model with a fully connected decoder can...