Introducing Amazon SageMaker Inference

In Chapter 6, you learned about SageMaker training and we dove into the details of the various options available to you when you train a model on SageMaker. Just like training, SageMaker provides a range of options when it comes to deploying models and generating predictions. These are available to you via the SageMaker Inference component. It’s important to note that the training and inference portions of SageMaker are decoupled from one another. This allows you to choose SageMaker for training, inference, or both. These operations are available to you via the AWS SDK as well as a dedicated SageMaker Python SDK. For more details on the SageMaker Python SDK, see the following link: https://sagemaker.readthedocs.io/en/stable/.

Let us now look at the details of options available to deploy models using SageMaker.

Understanding real-time endpoint options on SageMaker

In this option, SageMaker provides a persistent endpoint where the...