Training a Transformer model with NeuralForecast

Now, we turn our attention to Transformer architectures that have been driving recent advances in various fields of artificial intelligence. In this recipe, we will show you how to train a vanilla Transformer using the NeuralForecast Python library.

Getting ready

Transformers have become a dominant architecture in the deep learning community, especially for natural language processing (NLP) tasks. Transformers have been adopted for various tasks beyond NLP, including time series forecasting.

Unlike traditional models that analyze time series data point by point in sequence, Transformers evaluate all time steps simultaneously. This approach is similar to observing an entire timeline at once, determining the significance of each moment in relation to others for a specific point in time.

At the core of the Transformer architecture is the attention mechanism. This mechanism calculates a weighted sum of input values, or values...