Understanding different deployment tools and host environments

There are different deployment tools in the MLOps technology stack that have different target use cases and host environments for deploying different model inference pipelines. In Chapter 7, Multi-Step Deep Learning Inference Pipeline, we learned the different inference scenarios and requirements and implemented a multi-step DL inference pipeline that can be deployed into a model hosting/serving environment. Now, we will learn how to deploy such a model to a few specific model hosting and serving environments. This is visualized in Figure 8.1 as follows:

Figure 8.1 – Using model deployment tools to deploy a model inference pipeline to a model hosting and serving environment

As can be seen from Figure 8.1, there can be different deployment tools for different model hosting and serving environments. Here, we list the three typical scenarios as follows:

• Batch inference at scale: If we...