You're reading from Pretrain Vision and Large Language Models in Python End-to-end techniques for building and deploying foundation models on AWS

Product type Paperback

Published in May 2023

Publisher Packt

ISBN-13 9781804618257

Length 258 pages

Edition 1st Edition

Languages

Python

Tools

AWS

Concepts

GPT/LLMs

Author (1):

Emily Webber

Preface

1. Part 1: Before Pretraining

2. Chapter 1: An Introduction to Pretraining Foundation Models FREE CHAPTER

3. Chapter 2: Dataset Preparation: Part One

4. Chapter 3: Model Preparation

5. Part 2: Configure Your Environment

6. Chapter 4: Containers and Accelerators on the Cloud

7. Chapter 5: Distribution Fundamentals

8. Chapter 6: Dataset Preparation: Part Two, the Data Loader

9. Part 3: Train Your Model

10. Chapter 7: Finding the Right Hyperparameters

11. Chapter 8: Large-Scale Training on SageMaker

12. Chapter 9: Advanced Training Concepts

13. Part 4: Evaluate Your Model

14. Chapter 10: Fine-Tuning and Evaluating

15. Chapter 11: Detecting, Mitigating, and Monitoring Bias

16. Chapter 12: How to Deploy Your Model

17. Part 5: Deploy Your Model

18. Chapter 13: Prompt Engineering

19. Chapter 14: MLOps for Vision and Language

20. Chapter 15: Future Trends in Pretraining Foundation Models

21. Index

22. Other Books You May Enjoy

References

LLaMA: Open and Efficient Foundation Language Models: https://arxiv.org/pdf/2302.13971.pdf
Lambda quotas: https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html
Knowledge Distillation: A Survey: https://arxiv.org/pdf/2006.05525.pdf
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter: https://arxiv.org/pdf/1910.01108.pdf
QUANTIZATION: https://pytorch.org/docs/stable/quantization.html
Achieve hyperscale performance for model serving using NVIDIA Triton Inference Server on Amazon SageMaker: https://aws.amazon.com/blogs/machine-learning/achieve-hyperscale-performance-for-model-serving-using-nvidia-triton-inference-server-on-amazon-sagemaker/
Deploy BLOOM-176B and OPT-30B on Amazon SageMaker with large model inference Deep Learning Containers and DeepSpeed: https://aws.amazon.com/blogs/machine-learning/deploy-bloom-176b-and-opt-30b-on-amazon-sagemaker-with-large-model-inference-deep-learning-containers-and...