Training the NMT

Now that we have defined the NMT architecture and preprocessed the training data, it is quite straightforward to train the model. Here, we will define and illustrate (see Figure 9.15) the exact process used for training:

Figure 9.15: The training procedure for NMT

For the model training, we’re going to define a custom training loop, as there is a special metric we’d like to track. Unfortunately, this metric is not a readily available TensorFlow metric. But before that, there are several utility functions we need to define:

def prepare_data(de_lookup_layer, train_xy, valid_xy, test_xy):
    """ Create a data dictionary from the dataframes containing data 
    """
    
    data_dict = {}
    for label, data_xy in zip(['train', 'valid', 'test'], [train_xy, 
    valid_xy, test_xy]):
        
        data_x, data_y = data_xy
        en_inputs = data_x
        de_inputs = data_y[...