What do you get with Print?

Instant access to your digital eBook copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

AI Assistant (beta) to help accelerate your learning

A Primer on Transformers

The transformer is one of the most popular state-of-the-art deep learning architectures that is mostly used for natural language processing (NLP) tasks. Ever since the advent of the transformer, it has replaced RNN and LSTM for various tasks. Several new NLP models, such as BERT, GPT, and T5, are based on the transformer architecture. In this chapter, we will look into the transformer in detail and understand how it works.

We will begin the chapter by getting a basic idea of the transformer. Then, we will learn how the transformer uses encoder-decoder architecture for a language translation task. Following this, we will inspect how the encoder of the transformer works in detail by exploring each of the encoder components. After understanding the encoder, we will deep dive into the decoder and look into each of the decoder components in detail. At the...

Key benefits

Explore the encoder and decoder of the transformer model

Become well-versed with BERT along with ALBERT, RoBERTa, and DistilBERT

Discover how to pre-train and fine-tune BERT models for several NLP tasks

Description

BERT (bidirectional encoder representations from transformer) has revolutionized the world of natural language processing (NLP) with promising results. This book is an introductory guide that will help you get to grips with Google's BERT architecture. With a detailed explanation of the transformer architecture, this book will help you understand how the transformer’s encoder and decoder work. You’ll explore the BERT architecture by learning how the BERT model is pre-trained and how to use pre-trained BERT for downstream tasks by fine-tuning it for NLP tasks such as sentiment analysis and text summarization with the Hugging Face transformers library. As you advance, you’ll learn about different variants of BERT such as ALBERT, RoBERTa, and ELECTRA, and look at SpanBERT, which is used for NLP tasks like question answering. You'll also cover simpler and faster BERT variants based on knowledge distillation such as DistilBERT and TinyBERT. The book takes you through MBERT, XLM, and XLM-R in detail and then introduces you to sentence-BERT, which is used for obtaining sentence representation. Finally, you'll discover domain-specific BERT models such as BioBERT and ClinicalBERT, and discover an interesting variant called VideoBERT. By the end of this BERT book, you’ll be well-versed with using BERT and its variants for performing practical NLP tasks.

What you will learn

Understand the transformer model from the ground up

Find out how BERT works and pre-train it using masked language model (MLM) and next sentence prediction (NSP) tasks

Get hands-on with BERT by learning to generate contextual word and sentence embeddings

Fine-tune BERT for downstream tasks

Get to grips with ALBERT, RoBERTa, ELECTRA, and SpanBERT models

Get the hang of the BERT models based on knowledge distillation

Understand cross-lingual models such as XLM and XLM-R

Explore Sentence-BERT, VideoBERT, and BART

What do you get with Print?

Instant access to your digital eBook copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

AI Assistant (beta) to help accelerate your learning

Frequently bought together

$43.99

$54.99

Transformers for Natural Language Processing

$99.99

Total $ 198.97 Stars icon

Filter reviews by

All

Amazon verified reviews

Amazon Customer Jul 20, 2023

The simplest explanation of Transformers I ever found. I think Kdnuggets article on Transformers and this book provide a simple explanation of Transformers.

Amazon Verified review

ProTrader Jan 29, 2023

The first chapter “Primer on Transformers” is more or less copy/pasted from a well-regarded blogger in the NLP space, just that this version is 10x less understandable.The chapter on fine-tuning BERT for a NER task is an absolute joke. The chapter literally says “let's learn how to fine-tune the pre-trained BERT model to perform NER”, followed by 10 generic sentences, without any code, and concludes. The lack of quality persists across all other chapters.The code formatting of the Kindle version is horrific. The code is written as standard text, without any formatting while being aligned like a newspaper column.

Daniel Svoboda Jun 27, 2022

This book gives a clear and succinct overview to BERT, starting with the basics of Transformers and how they revolutionized the industry to the development of BERT and its revolution in being able to get embeddings of sentences along with other features.Normally, such topics are difficult, but this book gives a layman's overview for the budding big data engineer or data scientist. With technology rapidly evolving, such explanations will be welcome to people who want to stay current.

Colbert Philippe Jun 05, 2022

Excellent! A book on of the most active area of AI/ML. It's one of those books that is worth it weight in gold and even more.

Mohd Azam May 25, 2022

This is best book for advance NLP tasks. BERT is not more complex … recommended for NLP developers

Getting Started with Google BERT: Build and train state-of-the-art natural language processing models using BERT

What do you get with Print?