Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds
Pretrain Vision and Large Language Models in Python
Pretrain Vision and Large Language Models in Python

Pretrain Vision and Large Language Models in Python: End-to-end techniques for building and deploying foundation models on AWS

Arrow left icon
Profile Icon Emily Webber
Arrow right icon
$51.99
Full star icon Full star icon Full star icon Full star icon Half star icon 4.4 (20 Ratings)
Paperback May 2023 258 pages 1st Edition
eBook
$9.99 $41.99
Paperback
$51.99
Arrow left icon
Profile Icon Emily Webber
Arrow right icon
$51.99
Full star icon Full star icon Full star icon Full star icon Half star icon 4.4 (20 Ratings)
Paperback May 2023 258 pages 1st Edition
eBook
$9.99 $41.99
Paperback
$51.99
eBook
$9.99 $41.99
Paperback
$51.99

What do you get with Print?

Product feature icon Instant access to your digital copy whilst your Print order is Shipped
Product feature icon Colour book shipped to your preferred address
Product feature icon Redeem a companion digital copy on all Print orders
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Product feature icon AI Assistant (beta) to help accelerate your learning
Modal Close icon
Payment Processing...
tick Completed

Shipping Address

Billing Address

Shipping Methods
Table of content icon View table of contents Preview book icon Preview Book

Pretrain Vision and Large Language Models in Python

Left arrow icon Right arrow icon
Download code icon Download Code

Key benefits

  • Learn to develop, train, tune, and apply foundation models with optimized end-to-end pipelines
  • Explore large-scale distributed training for models and datasets with AWS and SageMaker examples
  • Evaluate, deploy, and operationalize your custom models with bias detection and pipeline monitoring

Description

Foundation models have forever changed machine learning. From BERT to ChatGPT, CLIP to Stable Diffusion, when billions of parameters are combined with large datasets and hundreds to thousands of GPUs, the result is nothing short of record-breaking. The recommendations, advice, and code samples in this book will help you pretrain and fine-tune your own foundation models from scratch on AWS and Amazon SageMaker, while applying them to hundreds of use cases across your organization. With advice from seasoned AWS and machine learning expert Emily Webber, this book helps you learn everything you need to go from project ideation to dataset preparation, training, evaluation, and deployment for large language, vision, and multimodal models. With step-by-step explanations of essential concepts and practical examples, you’ll go from mastering the concept of pretraining to preparing your dataset and model, configuring your environment, training, fine-tuning, evaluating, deploying, and optimizing your foundation models. You will learn how to apply the scaling laws to distributing your model and dataset over multiple GPUs, remove bias, achieve high throughput, and build deployment pipelines. By the end of this book, you’ll be well equipped to embark on your own project to pretrain and fine-tune the foundation models of the future.

Who is this book for?

If you’re a machine learning researcher or enthusiast who wants to start a foundation modelling project, this book is for you. Applied scientists, data scientists, machine learning engineers, solution architects, product managers, and students will all benefit from this book. Intermediate Python is a must, along with introductory concepts of cloud computing. A strong understanding of deep learning fundamentals is needed, while advanced topics will be explained. The content covers advanced machine learning and cloud techniques, explaining them in an actionable, easy-to-understand way.

What you will learn

  • Find the right use cases and datasets for pretraining and fine-tuning
  • Prepare for large-scale training with custom accelerators and GPUs
  • Configure environments on AWS and SageMaker to maximize performance
  • Select hyperparameters based on your model and constraints
  • Distribute your model and dataset using many types of parallelism
  • Avoid pitfalls with job restarts, intermittent health checks, and more
  • Evaluate your model with quantitative and qualitative insights
  • Deploy your models with runtime improvements and monitoring pipelines
Estimated delivery fee Deliver to United States

Economy delivery 10 - 13 business days

Free $6.95

Premium delivery 6 - 9 business days

$21.95
(Includes tracking information)

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : May 31, 2023
Length: 258 pages
Edition : 1st
Language : English
ISBN-13 : 9781804618257
Category :
Languages :
Concepts :
Tools :

What do you get with Print?

Product feature icon Instant access to your digital copy whilst your Print order is Shipped
Product feature icon Colour book shipped to your preferred address
Product feature icon Redeem a companion digital copy on all Print orders
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Product feature icon AI Assistant (beta) to help accelerate your learning
Modal Close icon
Payment Processing...
tick Completed

Shipping Address

Billing Address

Shipping Methods
Estimated delivery fee Deliver to United States

Economy delivery 10 - 13 business days

Free $6.95

Premium delivery 6 - 9 business days

$21.95
(Includes tracking information)

Product Details

Publication date : May 31, 2023
Length: 258 pages
Edition : 1st
Language : English
ISBN-13 : 9781804618257
Category :
Languages :
Concepts :
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
$19.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
$199.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts
$279.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total $ 150.97
Building AI Applications with ChatGPT APIs
$46.99
Modern Generative AI with ChatGPT and OpenAI Models
$51.99
Pretrain Vision and Large Language Models in Python
$51.99
Total $ 150.97 Stars icon

Table of Contents

22 Chapters
Part 1: Before Pretraining Chevron down icon Chevron up icon
Chapter 1: An Introduction to Pretraining Foundation Models Chevron down icon Chevron up icon
Chapter 2: Dataset Preparation: Part One Chevron down icon Chevron up icon
Chapter 3: Model Preparation Chevron down icon Chevron up icon
Part 2: Configure Your Environment Chevron down icon Chevron up icon
Chapter 4: Containers and Accelerators on the Cloud Chevron down icon Chevron up icon
Chapter 5: Distribution Fundamentals Chevron down icon Chevron up icon
Chapter 6: Dataset Preparation: Part Two, the Data Loader Chevron down icon Chevron up icon
Part 3: Train Your Model Chevron down icon Chevron up icon
Chapter 7: Finding the Right Hyperparameters Chevron down icon Chevron up icon
Chapter 8: Large-Scale Training on SageMaker Chevron down icon Chevron up icon
Chapter 9: Advanced Training Concepts Chevron down icon Chevron up icon
Part 4: Evaluate Your Model Chevron down icon Chevron up icon
Chapter 10: Fine-Tuning and Evaluating Chevron down icon Chevron up icon
Chapter 11: Detecting, Mitigating, and Monitoring Bias Chevron down icon Chevron up icon
Chapter 12: How to Deploy Your Model Chevron down icon Chevron up icon
Part 5: Deploy Your Model Chevron down icon Chevron up icon
Chapter 13: Prompt Engineering Chevron down icon Chevron up icon
Chapter 14: MLOps for Vision and Language Chevron down icon Chevron up icon
Chapter 15: Future Trends in Pretraining Foundation Models Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon
Other Books You May Enjoy Chevron down icon Chevron up icon

Customer reviews

Top Reviews
Rating distribution
Full star icon Full star icon Full star icon Full star icon Half star icon 4.4
(20 Ratings)
5 star 70%
4 star 15%
3 star 5%
2 star 5%
1 star 5%
Filter icon Filter
Top Reviews

Filter reviews by




alla Jul 07, 2023
Full star icon Full star icon Full star icon Full star icon Full star icon 5
This compelling guide is a treasure for machine learning engineers aspiring to dive deep into working within the AWS ecosystem.It proficiently lays out the practices of model algorithms and development, from selecting the ideal design and training dataset to deploying on AWS.With the author's expertise and insightful predictions, readers are navigated through the wonders of Sagemaker, model evaluation, and future trends.Highly recommended for those ready to embark on this awesome journey.
Amazon Verified review Amazon
Steven Fernandes Aug 11, 2023
Full star icon Full star icon Full star icon Full star icon Full star icon 5
In this comprehensive guide, readers are taken on a journey through the intricacies of machine learning model optimization, starting with the selection of ideal use cases and datasets for pretraining and fine-tuning. The book stands out in its detailed discussions on harnessing the power of custom accelerators and GPUs for large-scale training. It provides a hands-on tutorial on configuring AWS and SageMaker environments for optimal performance, which many practitioners will find invaluable. One of the key highlights is the chapter on hyperparameter selection, where the relationship between model constraints and optimal settings is delved into with clarity. The sections on model and dataset distribution using various parallelism techniques offer insights into efficient data handling. Practical advice, such as how to navigate challenges like job restarts and health checks, ensures readers are well-equipped to handle real-world scenarios. Finally, the guide culminates with thorough methods to evaluate and deploy models, emphasizing both quantitative and qualitative insights, and the importance of effective runtime improvements and monitoring. A must-read for anyone keen on mastering the art of machine learning at scale.
Amazon Verified review Amazon
Deep P. Jul 31, 2023
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Pretrain Vision and Large Language Models in Python is a book by Emily Webber that teaches you how to train and deploy foundation models on AWS. The book covers a wide range of topics, from the basics of pretraining to the deployment of models on SageMaker.The book is well-written and easy to follow, and it includes a number of helpful resources, such as code samples, tutorials, and links to additional information. The author does a great job of explaining complex concepts in a clear and concise way, and she provides plenty of examples to help readers understand how to train and deploy foundation models on AWS.One of the things I really liked about this book is that it goes beyond just explaining the concepts. The author also provides guidance on how to use these concepts to solve real-world problems. For example, she shows how to use foundation models to build applications that can automatically generate text, translate languages, and answer questions.Overall, I thought Pretrain Vision and Large Language Models in Python was an excellent resource for anyone who wants to learn more about foundation models or who wants to use these models to build their own applications. The book is well-written, informative, and easy to follow. I highly recommend it.
Amazon Verified review Amazon
Om S Jun 05, 2023
Full star icon Full star icon Full star icon Full star icon Full star icon 5
As a long-time admirer of Emily Webber and an enthusiast of SageMaker and Large Language Models (LLMs), I've been eagerly anticipating the release of this book. As a regular user of AWS, I can confidently state that this book is a substantial contribution to the advanced field of Machine Learning (ML).In the fast-paced world of ML, where LLMs are at the forefront of innovation, keeping up with the rapid advancements can be challenging. However, Emily's book, announced just before the LLM boom, provides a solid foundation that enables you to navigate these developments with ease. It has proven to be a timely resource for those keen on understanding and leveraging the power of LLMs.Book Summary:Comprehensive Coverage: The book offers an in-depth exploration of training vision and large language models, covering all stages from project ideation, dataset preparation, training, evaluation, to deployment for large language, vision, and multimodal models.Expert Guidance: Authored by Emily Webber, a seasoned AWS and machine learning expert, the book provides industry-expert guidance and practical advice, making it a valuable resource for both beginners and experienced practitioners.Practical Approach: The book is replete with practical examples and code samples that help readers understand how to pretrain and fine-tune their own foundation models on AWS and Amazon SageMaker.Bias Detection: A unique feature of the book is its focus on bias detection and pipeline monitoring, which are critical aspects of model development and deployment.Advanced Topics: The book delves into advanced topics like large-scale distributed training, hyperparameter selection, and model distribution, providing readers with a deep understanding of these complex areas.Future Trends: The final chapter on future trends in pretraining foundation models gives readers a glimpse into what's next in the field, keeping them ahead of the curve.In conclusion, if you're looking to ride the wave of LLMs and want to do so using AWS, this book is a must-read. It's more than just a guide; not a beginner's book!!! It's a comprehensive resource that empowers you to navigate the fast-paced world of ML with confidence and proficiency. Emily Webber's expertise shines through each page, making this book an invaluable asset for anyone in the field. As LLMs continue to evolve and revolutionize various sectors, this book stands as a testament to their transformative potential and a guide for those looking to be part of this exciting journey. This is just the start..... Transformers......... !!!
Amazon Verified review Amazon
Pratik Bhavsar Oct 03, 2023
Full star icon Full star icon Full star icon Full star icon Full star icon 5
If you've ever wondered about the intricacies of building massive language models, this book is an absolute must-read. It's jam-packed with invaluable insights on how to harness the full power of AWS to supercharge your language model workflows. 💡While there's no shortage of high-level content on the subject, what sets this book apart is its deep dive into the nitty-gritty, low-level details that are often elusive but oh-so-critical in this field.
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

What is the digital copy I get with my Print order? Chevron down icon Chevron up icon

When you buy any Print edition of our Books, you can redeem (for free) the eBook edition of the Print Book you’ve purchased. This gives you instant access to your book when you make an order via PDF, EPUB or our online Reader experience.

What is the delivery time and cost of print book? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela
What is custom duty/charge? Chevron down icon Chevron up icon

Customs duty are charges levied on goods when they cross international borders. It is a tax that is imposed on imported goods. These duties are charged by special authorities and bodies created by local governments and are meant to protect local industries, economies, and businesses.

Do I have to pay customs charges for the print book order? Chevron down icon Chevron up icon

The orders shipped to the countries that are listed under EU27 will not bear custom charges. They are paid by Packt as part of the order.

List of EU27 countries: www.gov.uk/eu-eea:

A custom duty or localized taxes may be applicable on the shipment and would be charged by the recipient country outside of the EU27 which should be paid by the customer and these duties are not included in the shipping charges been charged on the order.

How do I know my custom duty charges? Chevron down icon Chevron up icon

The amount of duty payable varies greatly depending on the imported goods, the country of origin and several other factors like the total invoice amount or dimensions like weight, and other such criteria applicable in your country.

For example:

  • If you live in Mexico, and the declared value of your ordered items is over $ 50, for you to receive a package, you will have to pay additional import tax of 19% which will be $ 9.50 to the courier service.
  • Whereas if you live in Turkey, and the declared value of your ordered items is over € 22, for you to receive a package, you will have to pay additional import tax of 18% which will be € 3.96 to the courier service.
How can I cancel my order? Chevron down icon Chevron up icon

Cancellation Policy for Published Printed Books:

You can cancel any order within 1 hour of placing the order. Simply contact customercare@packt.com with your order details or payment transaction id. If your order has already started the shipment process, we will do our best to stop it. However, if it is already on the way to you then when you receive it, you can contact us at customercare@packt.com using the returns and refund process.

Please understand that Packt Publishing cannot provide refunds or cancel any order except for the cases described in our Return Policy (i.e. Packt Publishing agrees to replace your printed book because it arrives damaged or material defect in book), Packt Publishing will not accept returns.

What is your returns and refunds policy? Chevron down icon Chevron up icon

Return Policy:

We want you to be happy with your purchase from Packtpub.com. We will not hassle you with returning print books to us. If the print book you receive from us is incorrect, damaged, doesn't work or is unacceptably late, please contact Customer Relations Team on customercare@packt.com with the order number and issue details as explained below:

  1. If you ordered (eBook, Video or Print Book) incorrectly or accidentally, please contact Customer Relations Team on customercare@packt.com within one hour of placing the order and we will replace/refund you the item cost.
  2. Sadly, if your eBook or Video file is faulty or a fault occurs during the eBook or Video being made available to you, i.e. during download then you should contact Customer Relations Team within 14 days of purchase on customercare@packt.com who will be able to resolve this issue for you.
  3. You will have a choice of replacement or refund of the problem items.(damaged, defective or incorrect)
  4. Once Customer Care Team confirms that you will be refunded, you should receive the refund within 10 to 12 working days.
  5. If you are only requesting a refund of one book from a multiple order, then we will refund you the appropriate single item.
  6. Where the items were shipped under a free shipping offer, there will be no shipping costs to refund.

On the off chance your printed book arrives damaged, with book material defect, contact our Customer Relation Team on customercare@packt.com within 14 days of receipt of the book with appropriate evidence of damage and we will work with you to secure a replacement copy, if necessary. Please note that each printed book you order from us is individually made by Packt's professional book-printing partner which is on a print-on-demand basis.

What tax is charged? Chevron down icon Chevron up icon

Currently, no tax is charged on the purchase of any print book (subject to change based on the laws and regulations). A localized VAT fee is charged only to our European and UK customers on eBooks, Video and subscriptions that they buy. GST is charged to Indian customers for eBooks and video purchases.

What payment methods can I use? Chevron down icon Chevron up icon

You can pay with the following card types:

  1. Visa Debit
  2. Visa Credit
  3. MasterCard
  4. PayPal
What is the delivery time and cost of print books? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela
Modal Close icon
Modal Close icon