You're reading from Amazon SageMaker Best Practices Proven tips and tricks to build successful machine learning solutions on Amazon SageMaker

Product type Paperback

Published in Sep 2021

Publisher Packt

ISBN-13 9781801070522

Length 348 pages

Edition 1st Edition

Languages

Python

Tools

Amazon SimpleDB

Concepts

Machine Learning

Authors (3):

Randy DeFauw

Shelbee Eigenbrode

Sireesha Muppala

View More author details

Table of Contents (20) Chapters

Preface

1. Section 1: Processing Data at Scale

2. Chapter 1: Amazon SageMaker Overview FREE CHAPTER

3. Chapter 2: Data Science Environments

4. Chapter 3: Data Labeling with Amazon SageMaker Ground Truth

5. Chapter 4: Data Preparation at Scale Using Amazon SageMaker Data Wrangler and Processing

6. Chapter 5: Centralized Feature Repository with Amazon SageMaker Feature Store

7. Section 2: Model Training Challenges

8. Chapter 6: Training and Tuning at Scale

9. Chapter 7: Profile Training Jobs with Amazon SageMaker Debugger

10. Section 3: Manage and Monitor Models

11. Chapter 8: Managing Models at Scale Using a Model Registry

12. Chapter 9: Updating Production Models Using Amazon SageMaker Endpoint Production Variants

13. Chapter 10: Optimizing Model Hosting and Inference Costs

14. Chapter 11: Monitoring Production Models with Amazon SageMaker Model Monitor and Clarify

15. Section 4: Automate and Operationalize Machine Learning

16. Chapter 12: Machine Learning Automated Workflows

17. Chapter 13:Well-Architected Machine Learning with Amazon SageMaker

18. Chapter 14: Managing SageMaker Features across Accounts

19. Other Books You May Enjoy

Summary

In this chapter, we tackled feature engineering for a large (~ 500 GB) dataset. We looked at challenges including scalability, bias, and explainability. We saw how to use SageMaker Data Wrangler, Clarify, and Processing jobs to explore and prepare data.

While there are many ways to use these tools, we recommend using Data Wrangler for interactive exploration of small to mid-sized datasets. For processing large datasets in their entirety, switch to programmatic use of processing jobs using the Spark framework to take advantage of parallel processing. (At the time of writing, Data Wrangler does not support running on multiple instances, but you can run a processing job on multiple instances.) You can always export a Data Wrangler flow as a starting point.

If your dataset is many terabytes, consider running a Spark job directly in EMR or Glue and invoking SageMaker using the SageMaker Spark SDK. EMR and Glue have optimized Spark runtimes and more efficient integration with...