You're reading from Machine Learning Model Serving Patterns and Best Practices A definitive guide to deploying, monitoring, and providing accessibility to ML models in production

Product type Paperback

Published in Dec 2022

Publisher Packt

ISBN-13 9781803249902

Length 336 pages

Edition 1st Edition

Languages

Python

Tools

AWS

Concepts

Machine Learning

Author (1):

Md Johirul Islam

View More author details

Table of Contents (22) Chapters

Preface

1. Part 1:Introduction to Model Serving

2. Chapter 1: Introducing Model Serving FREE CHAPTER

3. Chapter 2: Introducing Model Serving Patterns

4. Part 2:Patterns and Best Practices of Model Serving

5. Chapter 3: Stateless Model Serving

6. Chapter 4: Continuous Model Evaluation

7. Chapter 5: Keyed Prediction

8. Chapter 6: Batch Model Serving

9. Chapter 7: Online Learning Model Serving

10. Chapter 8: Two-Phase Model Serving

11. Chapter 9: Pipeline Pattern Model Serving

12. Chapter 10: Ensemble Model Serving Pattern

13. Chapter 11: Business Logic Pattern

14. Part 3:Introduction to Tools for Model Serving

15. Chapter 12: Exploring TensorFlow Serving

16. Chapter 13: Using Ray Serve

17. Chapter 14: Using BentoML

18. Part 4:Exploring Cloud Solutions

19. Chapter 15: Serving ML Models using a Fully Managed AWS Sagemaker Cloud Solution

20. Index

Why subscribe?

21. Other Books You May Enjoy

Introducing Ray Serve

Ray Serve is a framework-agnostic model-serving library. It is scalable and creates inference APIs on your behalf. Some of the key concepts in Ray Serve are as follows:

Deployment
ServeHandle
Ingress deployment

We will look at each of these in the following sections.

Deployment

A deployment contains the business logic and the ML model that will be served. To define a deployment, the @serve.deployment decorator is used. For example, let’s take a look at the following code snippet, which shows a very basic deployment that will return whatever message is passed by the user as a payload:

@serve.deployment class MyFirstDeployment:  # Take the message to return as an argument to the constructor.  def __init__(self, msg):      self.msg = msg   def __call__(self):      return self.msg my_first_deployment = MyFirstDeployment.bind("Hello...