You're reading from Mastering NLP from Foundations to LLMs Apply advanced rule-based techniques to LLMs and solve real-world business problems using Python

Product type Paperback

Published in Apr 2024

Publisher Packt

ISBN-13 9781804619186

Length 340 pages

Edition 1st Edition

Languages

Python

Concepts

Deep Learning

Authors (2):

Meysam Ghaffari

Lior Gazit

View More author details

Table of Contents (14) Chapters

Preface

1. Chapter 1: Navigating the NLP Landscape: A Comprehensive Introduction

2. Chapter 2: Mastering Linear Algebra, Probability, and Statistics for Machine Learning and NLP FREE CHAPTER

3. Chapter 3: Unleashing Machine Learning Potentials in Natural Language Processing

4. Chapter 4: Streamlining Text Preprocessing Techniques for Optimal NLP Performance

5. Chapter 5: Empowering Text Classification: Leveraging Traditional Machine Learning Techniques

6. Chapter 6: Text Classification Reimagined: Delving Deep into Deep Learning Language Models

7. Chapter 7: Demystifying Large Language Models: Theory, Design, and Langchain Implementation

8. Chapter 8: Accessing the Power of Large Language Models: Advanced Setup and Integration with RAG

9. Chapter 9: Exploring the Frontiers: Advanced Applications and Innovations Driven by LLMs

10. Chapter 10: Riding the Wave: Analyzing Past, Present, and Future Trends Shaped by LLMs and AI

11. Chapter 11: Exclusive Industry Insights: Perspectives and Predictions from World Class Experts

12. Index

Why subscribe?

13. Other Books You May Enjoy

Example designs of state-of-the-art LLMs

In this part, we are going to dig more into the design and architecture of some of the newest LLMs at the time of writing this book.

GPT-3.5 and ChatGPT

The core of ChatGPT is a Transformer, a type of model architecture that uses self-attention mechanisms to weigh the relevance of different words in the input when making predictions. It allows the model to consider the full context of the input when generating a response.

The GPT model

ChatGPT is based on the GPT version of the Transformer. The GPT models are trained to predict the next word in a sequence of words, given all the previous words. They process text from left to right (unidirectional context), which makes them well-suited for text generation tasks. For instance, GPT-3, one of the versions of GPT on which ChatGPT is based, contains 175 billion parameters.