You're reading from Machine Learning Model Serving Patterns and Best Practices A definitive guide to deploying, monitoring, and providing accessibility to ML models in production

Product type Paperback

Published in Dec 2022

Publisher Packt

ISBN-13 9781803249902

Length 336 pages

Edition 1st Edition

Languages

Python

Tools

AWS

Concepts

Machine Learning

Author (1):

Md Johirul Islam

Preface

1. Part 1:Introduction to Model Serving

2. Chapter 1: Introducing Model Serving FREE CHAPTER

3. Chapter 2: Introducing Model Serving Patterns

4. Part 2:Patterns and Best Practices of Model Serving

5. Chapter 3: Stateless Model Serving

6. Chapter 4: Continuous Model Evaluation

7. Chapter 5: Keyed Prediction

8. Chapter 6: Batch Model Serving

9. Chapter 7: Online Learning Model Serving

10. Chapter 8: Two-Phase Model Serving

11. Chapter 9: Pipeline Pattern Model Serving

12. Chapter 10: Ensemble Model Serving Pattern

13. Chapter 11: Business Logic Pattern

14. Part 3:Introduction to Tools for Model Serving

15. Chapter 12: Exploring TensorFlow Serving

16. Chapter 13: Using Ray Serve

17. Chapter 14: Using BentoML

18. Part 4:Exploring Cloud Solutions

19. Chapter 15: Serving ML Models using a Fully Managed AWS Sagemaker Cloud Solution

20. Index

21. Other Books You May Enjoy

Limitations of batch serving

Batch serving is essential in today’s world of big data. However, it has the following limitations:

Scheduling the jobs is hard: Scheduling periodic batch jobs is sometimes complicated. As we have seen, during scheduling, the paths expected by the cron expression need to be given carefully. Mostly, cron expressions expect absolute paths. The scheduled jobs may also introduce a single point of failure. If somehow it fails to run on schedule, we might not have the latest inferences, causing a bad customer experience.
Growth of data will make training slow: If the data grows, the training may gradually take more time. For example, the time needed to train a model with 10 MB of data will not be the same as the time needed to train a model with 10 GB of data. Therefore, we need to take care of this scenario. In most cases, we can discard old data, as it will become stale. Then, the question arises, how old is the data when we consider it stale...