Preface

We've seen big changes in Natural Language Processing (NLP) over the last 20 years. During this time, we have experienced different paradigms and finally entered a new era dominated by the magical transformer architecture. This deep learning architecture has come about by inheriting many approaches. Contextual word embeddings, multi-head self-attention, positional encoding, parallelizable architectures, model compression, transfer learning, and cross-lingual models are among those approaches. Starting with the help of various neural-based NLP approaches, the transformer architecture gradually evolved into an attention-based encoder-decoder architecture and continues to evolve to this day. Now, we are seeing new successful variants of this architecture in the literature. Great models have emerged that use only the encoder part of it, such as BERT, or only the decoder part of it, such as GPT.

Throughout the book, we will touch on these NLP approaches and will be able to work with transformer models easily thanks to the Transformers library from the Hugging Face community. We will provide the solutions step by step to a wide variety of NLP problems, ranging from summarization to question-answering. We will see that we can achieve state-of-the-art results with the help of transformers.