You're reading from Modern Time Series Forecasting with Python Explore industry-ready time series forecasting using modern machine learning and deep learning

Product type Paperback

Published in Nov 2022

Publisher Packt

ISBN-13 9781803246802

Length 552 pages

Edition 1st Edition

Languages

Python

Tools

Time Series Forecasting

Concepts

Data Science

Author (1):

Manu Joseph

View More author details

Table of Contents (26) Chapters

Preface

1. Part 1 – Getting Familiar with Time Series

2. Chapter 1: Introducing Time Series FREE CHAPTER

3. Chapter 2: Acquiring and Processing Time Series Data

4. Chapter 3: Analyzing and Visualizing Time Series Data

5. Chapter 4: Setting a Strong Baseline Forecast

6. Part 2 – Machine Learning for Time Series

7. Chapter 5: Time Series Forecasting as Regression

8. Chapter 6: Feature Engineering for Time Series Forecasting

9. Chapter 7: Target Transformations for Time Series Forecasting

10. Chapter 8: Forecasting Time Series with Machine Learning Models

11. Chapter 9: Ensembling and Stacking

12. Chapter 10: Global Forecasting Models

13. Part 3 – Deep Learning for Time Series

14. Chapter 11: Introduction to Deep Learning

15. Chapter 12: Building Blocks of Deep Learning for Time Series

16. Chapter 13: Common Modeling Patterns for Time Series

17. Chapter 14: Attention and Transformers for Time Series

18. Chapter 15: Strategies for Global Deep Learning Forecasting Models

19. Chapter 16: Specialized Deep Learning Architectures for Forecasting

20. Part 4 – Mechanics of Forecasting

21. Chapter 17: Multi-Step Forecasting

22. Chapter 18: Evaluating Forecasts – Forecast Metrics

23. Chapter 19: Evaluating Forecasts – Validation Strategies

24. Index

Why subscribe?

25. Other Books You May Enjoy

Forecasting with Transformers

For some continuity, we will continue with the same household we were forecasting with RNNs and RNNs with attention.

Notebook alert

To follow along with the complete code, use the notebook named 03-Transformers.ipynb in the Chapter14 folder and the code in the src folder.

Although we learned about the vanilla Transformer as a model with an encoder-decoder architecture, it was really designed for language translation tasks. In language translation, the source sequence and target sequence are quite different, and therefore the encoder-decoder architecture made sense. But soon after, researchers figured out that using the decoder part of the Transformer alone does well. It is called a decoder-only Transformer in literature. The naming is a bit confusing because if you think about it, the decoder is different from the encoder in two things—masked self-attention and encoder-decoder attention. So, in a decoder-only Transformer, how do we have...