To get the most out of this book

• Most of the programs in the book are Colaboratory notebooks. All you will need is a free Google Gmail account, and you will be able to run the notebooks on Google Colaboratory's free VM.
• You will need Python installed on your machine for some of the educational programs.
• Take the necessary time to read Chapter 1, Getting Started with the Model Architecture of the Transformer. It contains the description of the Original Transformer, which is built from building blocks that will be implemented throughout the book. If you find it difficult, then pick up the general intuitive ideas out of the chapter. You can then go back to this chapter when you feel more comfortable with transformers after a few chapters.
• After reading each chapter, consider how you could implement transformers for your customers or use them to move up in your career with novel ideas.

Download the example code files

The code bundle for the book is hosted on GitHub at https://github.com/PacktPublishing/Transformers-for-Natural-Language-Processing. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that contains color images of the screenshots/diagrams used in this book. You can download it here: https://static.packt-cdn.com/downloads/9781800565791_ColorImages.pdf.

Conventions used

There are several text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. For example, "However, if you wish to explore the code, you will find it in the Google Colaboratory positional_encoding.ipynb notebook and the text.txt file in this chapter's GitHub repository."

A block of code is set as follows:

import numpy as np
from scipy.special import softmax

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

The black cat sat on the couch and the brown dog slept on the rug.

Any command-line input or output is written as follows:

[[0.99987495]] word similarity
[[0.8600013]] positional encoding vector similarity
[[0.9627094]] final positional encoding similarity

Bold: Indicates a new term, an important word, or words that you see on the screen, for example, in menus or dialog boxes, also appear in the text like this. For example: "In our case, we are looking for t5-large, a t5-large model we can smoothly run in Google Colaboratory."

Warnings or important notes appear like this.

Tips and tricks appear like this.