You're reading from Pretrain Vision and Large Language Models in Python End-to-end techniques for building and deploying foundation models on AWS

Product type Paperback

Published in May 2023

Publisher Packt

ISBN-13 9781804618257

Length 258 pages

Edition 1st Edition

Languages

Python

Tools

AWS

Concepts

GPT/LLMs

Author (1):

Emily Webber

View More author details

Table of Contents (23) Chapters

Preface

1. Part 1: Before Pretraining

2. Chapter 1: An Introduction to Pretraining Foundation Models FREE CHAPTER

3. Chapter 2: Dataset Preparation: Part One

4. Chapter 3: Model Preparation

5. Part 2: Configure Your Environment

6. Chapter 4: Containers and Accelerators on the Cloud

7. Chapter 5: Distribution Fundamentals

8. Chapter 6: Dataset Preparation: Part Two, the Data Loader

9. Part 3: Train Your Model

10. Chapter 7: Finding the Right Hyperparameters

11. Chapter 8: Large-Scale Training on SageMaker

12. Chapter 9: Advanced Training Concepts

13. Part 4: Evaluate Your Model

14. Chapter 10: Fine-Tuning and Evaluating

15. Chapter 11: Detecting, Mitigating, and Monitoring Bias

16. Chapter 12: How to Deploy Your Model

17. Part 5: Deploy Your Model

18. Chapter 13: Prompt Engineering

19. Chapter 14: MLOps for Vision and Language

20. Chapter 15: Future Trends in Pretraining Foundation Models

21. Index

Why subscribe?

22. Other Books You May Enjoy

The future of pretraining

In a full roundabout, let’s take a look at some skeptical trends that provide a sense of caution and critical evaluation of the recent goldrush in foundation models. One of the most prominent examples is Geoffrey Hinton quitting Google (29) to warn about the dangers of AI development, calling for a 6-month moratorium (30) on all foundation model research.

Personally, I think this pause fails to recognize what drove the hype in the first place: the invisible hand of AI economics. Landing high-paying tech jobs is not easy; getting and maintaining visas for the US and similar countries is non-trivial. Growing your career in an industry with some of the most intelligent and driven people, with ground-shattering fast-paced changes is anything but simple. Foundation models and all improvements in technology evolve because humans need to prove themselves, grow their careers, and provide for their families. Asking to pause experiments on large-scale training...