Introducing Model Serving

While machine learning (ML) surprises us every day with new, stunning ideas and demos, a burning question remains: how can we make the model available to our users? Often, we see demos of models on different blogs, books, YouTube videos, and so on, and we remain hungry to use the models ourselves. This is where model serving comes into the picture. Model serving is how we make our models available for use.

In this chapter, we will learn the definition of model serving, the importance of model serving, the challenges that make model serving difficult, and how people currently serve models, and see some of the available tools used for model serving.

By the end of this chapter, we will understand what model serving is, why model serving is needed, what makes it different from traditional web serving, and how people currently deploy/serve models.

In this chapter, we are going to cover the following main topics:

• What is serving?
• What are models?
• What is model serving?
• Understanding the importance of model serving
• Challenges of serving models
• Using existing tools to serve models