You're reading from Pretrain Vision and Large Language Models in Python End-to-end techniques for building and deploying foundation models on AWS

Product type Paperback

Published in May 2023

Publisher Packt

ISBN-13 9781804618257

Length 258 pages

Edition 1st Edition

Languages

Python

Tools

AWS

Concepts

GPT/LLMs

Author (1):

Emily Webber

Preface

1. Part 1: Before Pretraining

2. Chapter 1: An Introduction to Pretraining Foundation Models FREE CHAPTER

3. Chapter 2: Dataset Preparation: Part One

4. Chapter 3: Model Preparation

5. Part 2: Configure Your Environment

6. Chapter 4: Containers and Accelerators on the Cloud

7. Chapter 5: Distribution Fundamentals

8. Chapter 6: Dataset Preparation: Part Two, the Data Loader

9. Part 3: Train Your Model

10. Chapter 7: Finding the Right Hyperparameters

11. Chapter 8: Large-Scale Training on SageMaker

12. Chapter 9: Advanced Training Concepts

13. Part 4: Evaluate Your Model

14. Chapter 10: Fine-Tuning and Evaluating

15. Chapter 11: Detecting, Mitigating, and Monitoring Bias

16. Chapter 12: How to Deploy Your Model

17. Part 5: Deploy Your Model

18. Chapter 13: Prompt Engineering

19. Chapter 14: MLOps for Vision and Language

20. Chapter 15: Future Trends in Pretraining Foundation Models

21. Index

22. Other Books You May Enjoy

Getting ready to use your accelerators

Let’s start with learning how to use your accelerators:

Step one: acquisition. You definitely can’t train a model on a GPU without first getting your hands on at least one of the GPUs. Fortunately, there are a few free options for you. One of my projects at Amazon was actually writing the original doc for this: SageMaker Studio Lab! Studio Lab is one way to run a free Jupyter Notebook server in the cloud. If you’d like to use a no-cost notebook environment on CPUs or GPUs, store your files, collaborate with others, and connect to AWS or any other service, Studio Lab is a great way to get started.
Step two: containers. Once you’re in a Jupyter notebook and are trying to run some example code, you’ll realize that everything hinges on installing the right packages. Even once you have the packages installed, connecting them to the GPU depends on the CUDA installation in your notebook. If the version of...