What do you get with Print?

Instant access to your digital eBook copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

AI Assistant (beta) to help accelerate your learning

A Primer on Transformers

The transformer is one of the most popular state-of-the-art deep learning architectures that is mostly used for natural language processing (NLP) tasks. Ever since the advent of the transformer, it has replaced RNN and LSTM for various tasks. Several new NLP models, such as BERT, GPT, and T5, are based on the transformer architecture. In this chapter, we will look into the transformer in detail and understand how it works.

We will begin the chapter by getting a basic idea of the transformer. Then, we will learn how the transformer uses encoder-decoder architecture for a language translation task. Following this, we will inspect how the encoder of the transformer works in detail by exploring each of the encoder components. After understanding the encoder, we will deep dive into the decoder and look into each of the decoder components in detail. At the...

Key benefits

Explore the encoder and decoder of the transformer model

Become well-versed with BERT along with ALBERT, RoBERTa, and DistilBERT

Discover how to pre-train and fine-tune BERT models for several NLP tasks

Description

BERT (bidirectional encoder representations from transformer) has revolutionized the world of natural language processing (NLP) with promising results. This book is an introductory guide that will help you get to grips with Google's BERT architecture. With a detailed explanation of the transformer architecture, this book will help you understand how the transformer’s encoder and decoder work. You’ll explore the BERT architecture by learning how the BERT model is pre-trained and how to use pre-trained BERT for downstream tasks by fine-tuning it for NLP tasks such as sentiment analysis and text summarization with the Hugging Face transformers library. As you advance, you’ll learn about different variants of BERT such as ALBERT, RoBERTa, and ELECTRA, and look at SpanBERT, which is used for NLP tasks like question answering. You'll also cover simpler and faster BERT variants based on knowledge distillation such as DistilBERT and TinyBERT. The book takes you through MBERT, XLM, and XLM-R in detail and then introduces you to sentence-BERT, which is used for obtaining sentence representation. Finally, you'll discover domain-specific BERT models such as BioBERT and ClinicalBERT, and discover an interesting variant called VideoBERT. By the end of this BERT book, you’ll be well-versed with using BERT and its variants for performing practical NLP tasks.

What you will learn

Understand the transformer model from the ground up

Find out how BERT works and pre-train it using masked language model (MLM) and next sentence prediction (NSP) tasks

Get hands-on with BERT by learning to generate contextual word and sentence embeddings

Fine-tune BERT for downstream tasks

Get to grips with ALBERT, RoBERTa, ELECTRA, and SpanBERT models

Get the hang of the BERT models based on knowledge distillation

Understand cross-lingual models such as XLM and XLM-R

Explore Sentence-BERT, VideoBERT, and BART

What do you get with Print?

Instant access to your digital eBook copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

AI Assistant (beta) to help accelerate your learning

Frequently bought together

R$245.99

R$306.99

Transformers for Natural Language Processing

R$557.99

Total R$ 1,110.97

Filter reviews by

All

Amazon verified reviews

Amazon Customer Feb 07, 2021

The book acts as a great resource for learning BERT. It covers so many different types of BERT and helps you to learn how to apply BERT for interesting use cases. It’s a perfect getting started guide for BERT. The writing is so simple, clear, to the point. The way one topic connects to another is so interesting. I can’t close the book after reading one chapter, the book keeps you so engaging.

Amazon Verified review

ani Feb 05, 2021

The book explains transformers and BERT in very detail. I’m awestruck with the way the author have explained the concepts in a seamlessly simple way possible. I really loved the narrative style of the book.I can’t believe someone explained BERT with so much of in-depth detail. The book covers lot of things which I was never aware of and many different types of BERT like tinyBERT, ELECTRA, Multilingual BERT, XLM-R, and many others. If you are not getting this then definitely you are missing out a greatest content on BERT ever.

Samuel de Zoete May 25, 2021

Makes BERT accessible for Data Scientists without a PhD. The explanations are clear and still enough depth that’s needed when start working with Transformers.

aditya Karampudi Feb 09, 2021

The book starts off with subtle introduction to multiple key concepts and slowly builds on the core methodologies of building NLP based neural networks. Over the years, neural networks have gone through multiple transformations, and yet the application of these architectures is limited because of the nature of data. The language is the flow of words and these sentences do not always follow a structured approach. This makes it hard to train models that can be intelligent to understand the words. The BERT tries to tackle this issue by predicting from both directions- left to right and right to left. The author tried to use multiple examples to illustrate the way NN are modeled and I thoroughly enjoyed reading this book. I recommend this book for every NLP and Deep Learning enthusiast.

Ashwini Mar 05, 2021

I liked this book about BERT. This is one of the great books on BERT that I have come across. The author takes detailed accounts of introduction and applications of BERT before explaining things in detail. I loved the fact that there are frameworks explaining how each and every topic works in BERT. The book is a bit mathy but great for people who want to understand things in details.

Getting Started with Google BERT: Build and train state-of-the-art natural language processing models using BERT

What do you get with Print?