Generating summaries

The critical thing to note while generating summaries is that a new inference loop will need to be built. Recall that teacher forcing was used during training, and the output of the Decoder was not used in predicting the next token. While generating summaries, we would like to use the generated tokens in predicting the next token. Since we would like to play with various input texts and generate summaries, we will use the code in the generating-summaries.ipynb IPython notebook. After importing and setting everything up, the tokenizer needs to be instantiated. The Setup Tokenization section of the notebook loads the tokenizers and sets up the vocabulary by adding start and end token IDs. Similar to when we loaded the data, the data encoding method is set up to encode the input articles.

Now, we must hydrate the model from the saved checkpoint. All of the model objects are created first:

BATCH_SIZE = 1  # for inference
embedding_dim = 128
units = 256...