Transformers for Natural Language Processing: Build, train, and fine-tune deep neural network architectures for NLP with Python, Hugging Face, and OpenAI's GPT-3, ChatGPT, and GPT-4 , Second Edition

Denis Rothman

€67.99

3.8 (28 Ratings)

Paperback Mar 2022 602 pages 2nd Edition

Denis Rothman

€67.99

3.8 (28 Ratings)

Paperback Mar 2022 602 pages 2nd Edition

What do you get with Print?

Instant access to your digital eBook copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

AI Assistant (beta) to help accelerate your learning

View table of contents

Preview Book

Transformers for Natural Language Processing

Getting Started with the Architecture of the Transformer Model

Language is the essence of human communication. Civilizations would never have been born without the word sequences that form language. We now mostly live in a world of digital representations of language. Our daily lives rely on NLP digitalized language functions: web search engines, emails, social networks, posts, tweets, smartphone texting, translations, web pages, speech-to-text on streaming sites for transcripts, text-to-speech on hotline services, and many more everyday functions.

Chapter 1, What are Transformers?, explained the limits of RNNs and the birth of cloud AI transformers taking over a fair share of design and development. The role of the Industry 4.0 developer is to understand the architecture of the original Transformer and the multiple transformer ecosystems that followed.

In December 2017, Google Brain and Google Research published the seminal Vaswani et al., Attention is All You Need paper...

Tranformer models in Hugging Face

Everything you saw in this chapter can be condensed in to a ready-to-use Hugging Face transformer model.

With Hugging Face, you can implement machine translation in three lines of code!

Open Multi_Head_Attention_Sub_Layer.ipynb in Google Colaboratory. Save the notebook in your Google Drive (make sure you have a Gmail account). Go to the two last cells.

We first ensure that Hugging Face transformers are installed:

!pip -q install transformers

The first cell imports the Hugging Face pipeline that contains several transformer usages:

#@title Retrieve pipeline of modules and choose English to French translation
from transformers import pipeline

We then implement the Hugging Face pipeline, which contains ready-to-use functions. In our case, to illustrate the Transformer model of this chapter, we activate the translator model and enter a sentence to translate from English to French:

translator = pipeline("translation_en_to_fr...

Download Code

Key benefits

Improve your productivity with OpenAI’s ChatGPT and GPT-4 from prompt engineering to creating and analyzing machine learning models
Pretrain a BERT-based model from scratch using Hugging Face
Fine-tune powerful transformer models, including OpenAI's GPT-3, to learn the logic of your data

Description

Transformers are...well...transforming the world of AI. There are many platforms and models out there, but which ones best suit your needs? Transformers for Natural Language Processing, 2nd Edition, guides you through the world of transformers, highlighting the strengths of different models and platforms, while teaching you the problem-solving skills you need to tackle model weaknesses. You'll use Hugging Face to pretrain a RoBERTa model from scratch, from building the dataset to defining the data collator to training the model. If you're looking to fine-tune a pretrained model, including GPT-3, then Transformers for Natural Language Processing, 2nd Edition, shows you how with step-by-step guides. The book investigates machine translations, speech-to-text, text-to-speech, question-answering, and many more NLP tasks. It provides techniques to solve hard language problems and may even help with fake news anxiety (read chapter 13 for more details). You'll see how cutting-edge platforms, such as OpenAI, have taken transformers beyond language into computer vision tasks and code creation using DALL-E 2, ChatGPT, and GPT-4. By the end of this book, you'll know how transformers work and how to implement them and resolve issues like an AI detective.

Who is this book for?

If you want to learn about and apply transformers to your natural language (and image) data, this book is for you. You'll need a good understanding of Python and deep learning and a basic understanding of NLP to benefit most from this book. Many platforms covered in this book provide interactive user interfaces, which allow readers with a general interest in NLP and AI to follow several chapters. And don't worry if you get stuck or have questions; this book gives you direct access to our AI/ML community to help guide you on your transformers journey!

What you will learn

Discover new techniques to investigate complex language problems
Compare and contrast the results of GPT-3 against T5, GPT-2, and BERT-based transformers
Carry out sentiment analysis, text summarization, casual speech analysis, machine translations, and more using TensorFlow, PyTorch, and GPT-3
Find out how ViT and CLIP label images (including blurry ones!) and create images from a sentence using DALL-E
Learn the mechanics of advanced prompt engineering for ChatGPT and GPT-4

Estimated delivery fee Deliver to Spain

Premium delivery 7 - 10 business days

€17.95

(Includes tracking information)

What do you get with Print?

Instant access to your digital eBook copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

AI Assistant (beta) to help accelerate your learning

Frequently bought together

Machine Learning with PyTorch and Scikit-Learn

€41.99

Deep Learning with TensorFlow and Keras – 3rd edition

€37.99

€67.99

Total € 147.97

FAQs

What is the delivery time and cost of print book?

Shipping Details

USA:

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.

Unfortunately, due to several restrictions, we are unable to ship to the following countries:

Afghanistan
American Samoa
Belarus
Brunei Darussalam
Central African Republic
The Democratic Republic of Congo
Eritrea
Guinea-bissau
Iran
Lebanon
Libiya Arab Jamahriya
Somalia
Sudan
Russian Federation
Syrian Arab Republic
Ukraine
Venezuela

What is custom duty/charge?

Customs duty are charges levied on goods when they cross international borders. It is a tax that is imposed on imported goods. These duties are charged by special authorities and bodies created by local governments and are meant to protect local industries, economies, and businesses.

Do I have to pay customs charges for the print book order?

The orders shipped to the countries that are listed under EU27 will not bear custom charges. They are paid by Packt as part of the order.

List of EU27 countries: www.gov.uk/eu-eea:

A custom duty or localized taxes may be applicable on the shipment and would be charged by the recipient country outside of the EU27 which should be paid by the customer and these duties are not included in the shipping charges been charged on the order.

How do I know my custom duty charges?

The amount of duty payable varies greatly depending on the imported goods, the country of origin and several other factors like the total invoice amount or dimensions like weight, and other such criteria applicable in your country.

For example:

If you live in Mexico, and the declared value of your ordered items is over $ 50, for you to receive a package, you will have to pay additional import tax of 19% which will be $ 9.50 to the courier service.
Whereas if you live in Turkey, and the declared value of your ordered items is over € 22, for you to receive a package, you will have to pay additional import tax of 18% which will be € 3.96 to the courier service.

How can I cancel my order?

Cancellation Policy for Published Printed Books:

You can cancel any order within 1 hour of placing the order. Simply contact customercare@packt.com with your order details or payment transaction id. If your order has already started the shipment process, we will do our best to stop it. However, if it is already on the way to you then when you receive it, you can contact us at customercare@packt.com using the returns and refund process.

Please understand that Packt Publishing cannot provide refunds or cancel any order except for the cases described in our Return Policy (i.e. Packt Publishing agrees to replace your printed book because it arrives damaged or material defect in book), Packt Publishing will not accept returns.

What is your returns and refunds policy?

Return Policy:

We want you to be happy with your purchase from Packtpub.com. We will not hassle you with returning print books to us. If the print book you receive from us is incorrect, damaged, doesn't work or is unacceptably late, please contact Customer Relations Team on customercare@packt.com with the order number and issue details as explained below:

If you ordered (eBook, Video or Print Book) incorrectly or accidentally, please contact Customer Relations Team on customercare@packt.com within one hour of placing the order and we will replace/refund you the item cost.
Sadly, if your eBook or Video file is faulty or a fault occurs during the eBook or Video being made available to you, i.e. during download then you should contact Customer Relations Team within 14 days of purchase on customercare@packt.com who will be able to resolve this issue for you.
You will have a choice of replacement or refund of the problem items.(damaged, defective or incorrect)
Once Customer Care Team confirms that you will be refunded, you should receive the refund within 10 to 12 working days.
If you are only requesting a refund of one book from a multiple order, then we will refund you the appropriate single item.
Where the items were shipped under a free shipping offer, there will be no shipping costs to refund.

On the off chance your printed book arrives damaged, with book material defect, contact our Customer Relation Team on customercare@packt.com within 14 days of receipt of the book with appropriate evidence of damage and we will work with you to secure a replacement copy, if necessary. Please note that each printed book you order from us is individually made by Packt's professional book-printing partner which is on a print-on-demand basis.

What tax is charged?

Currently, no tax is charged on the purchase of any print book (subject to change based on the laws and regulations). A localized VAT fee is charged only to our European and UK customers on eBooks, Video and subscriptions that they buy. GST is charged to Indian customers for eBooks and video purchases.

What payment methods can I use?

You can pay with the following card types:

Visa Debit
Visa Credit
MasterCard
PayPal

What is the delivery time and cost of print books?

Shipping Details

USA:

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Unfortunately, due to several restrictions, we are unable to ship to the following countries:

Afghanistan
American Samoa
Belarus
Brunei Darussalam
Central African Republic
The Democratic Republic of Congo
Eritrea
Guinea-bissau
Iran
Lebanon
Libiya Arab Jamahriya
Somalia
Sudan
Russian Federation
Syrian Arab Republic
Ukraine
Venezuela

Filter reviews by

All

Packt verified reviews

Amazon verified reviews

Gabe Rigall Apr 05, 2022

The media could not be loaded. BLUF: This intermediate-to-advanced text provides a no-holds-barred introduction to transformer architecture and application for NLP (and other) tasks. If you're looking to up your NLP game, this book is for you.PROS:- Helpful background information for computer scientists and data wizards alike- Plenty of graphics and in-depth explanations of the "black boxes" of transformers; especially helpful for the visual learner- Good mix of theory and application with a focus on the latterCONS:- Fairly advanced text; assumes the reader has a certain breadth of subject matter knowledge (definitely not for beginners)

Amazon Verified review

dr t May 04, 2022

Transformer models have powered recent NLP developments and have completely changed the way NLP problems are now approached. Rothman believes that Industry 4.0 professionals need to be aware of multiple approaches and understand that each has its own pros and cons. However, this book is not designed to explain every single transformer model out there. Instead, it tries to explain enough so that readers have enough knowledge to know how to tackle an NLP problem.There is much to like about this book. The book has 16 chapters and begins by explaining transformers, and exploring interesting ideas such as whether programming is now becoming a sub-domain of NLP. There are also fantastic, and very useful, practical examples on how to work with a Bert tokeniser, conditioning a GPT-2 model, question-answering, pre-training RoBERTa models from scratch, and training a tokeniser. The running NLP tasks online section is also useful. Given this readers own background, the chapter on detecting customer emotions to make predictions was fascinating, and frankly left this reader wanting more!. In todays world where XAI is vitally important, the chapter on interpreting black box transformer models, and the section on using BertViz to show visualisations of the activity of transformer models are key to understanding how models work and to interpret model behaviour.This is one of those books where reading is fine, and the questions section at end of each chapter is useful, but one must really do to gain maximum benefit. The book contain lots of Python code to follow but, to be clear, this book is not geared towards Python beginners - there are plenty of great Packt Python beginners books out there already - hence knowledge of NLP and Python is mandatory.In summary, this book cements Rothman's position as the #1 authority on Transformers, and is absolutely the go-to book for Transformers. Highly recommended and a must-buy for any serious NLP practitioner.

Prateek M. Jul 11, 2022

for NLP practitioners who do not want to skim through the entire internet to look for any concept, this is a very good book as it is concise and good in both theory as well as code

Derek Martin Apr 06, 2022

I've read anything I can get my hands on re: GPT-3/Huggingface. Prof Rothman has raised the ante for what should be considered acceptable discourse re: GPT-3/Transformers and the new NLP-driven world that we live in.Buy the book; it has a feel of an experienced mentor giving you the tools but more importantly, the judgement to navigate these new waters.

hawkinflight Mar 25, 2022

This book answers the questions: What are transformers? Why do we care? What do they help us do? Why and how are they better?The 16 chapters cover a lot of material - the author explains what transformers are and how they have evolved, discusses tools and methods that can be used to interpret black box transformer models, and presents applications of transformer models to a variety of tasks, mostly in natural language processing/understanding, but also in computer vision. It's very informative to read about the evolution, and nice to learn the areas where there are limitations. The generalization that has occurred is described.It's great that the book provides interactive ways of learning. There are hands-on coding opportunities via Google Colaboratory notebooks, and questions at the end of each chapter.It's fascinating to read that OpenAI trained a 175 billion parameter GPT-3 transformer model to be run on a supercomputer with 10k GPUs and 285k CPU cores.The book mentions Industry 4.0, the Fourth Industrial Revolution, that is, building on top of the digital revolution, and connecting everything to everything, everywhere. Transformers are described as filling a gap left by the limitations of Recurrent Neural Networks (RNNs) for automation in a fast-moving world.It's also interesting to read that transformers mark a new generation of ready-to-use artificial intelligence models; for example, Hugging Face and Google Brain provide AI with a few lines of code.It is mentioned that "learning all of the models is impossible, however, by deepening our knowledge of transformer models, we can understand a new model quickly."The final chapter discusses AI Copilots and machine interconnectivity to speedup transactions and boost productivity. It says, for example, that Microsoft and OpenAI have collaborated and produced a copilot that can write Python code with humans or for humans.

Transformers for Natural Language Processing: Build, train, and fine-tune deep neural network architectures for NLP with Python, Hugging Face, and OpenAI's GPT-3, ChatGPT, and GPT-4 , Second Edition

What do you get with Print?

Transformers for Natural Language Processing

Getting Started with the Architecture of the Transformer Model

The rise of the Transformer: Attention is All You Need

Training and performance

Tranformer models in Hugging Face

Summary

Questions

References

Join our book’s Discord space

Page 1 of 7

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Product Details

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

About the author

FAQs

Transformers for Natural Language Processing: Build, train, and fine-tune deep neural network architectures for NLP with Python, Hugging Face, and OpenAI's GPT-3, ChatGPT, and GPT-4 , Second Edition

What do you get with Print?

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

About the author

FAQs