SageMaker inference

Amazon, in collaboration with Hugging Face, created a nice way of deploying models using only a few lines of code. To deploy any model using SageMaker, you can simply visit the model page at Hugging Face and click the Amazon SageMaker button. Please also note that other methods such as Azure are also available:

Figure 14.9 – Amazon SageMaker button

This will give the related code to use SageMaker for inference but note that you need to set up the AWS SageMaker environment first. We did not include this part in the book because it is another topic entirely, but you can easily find it in the SageMaker documentation (https://aws.amazon.com/sagemaker/).

Let’s see how we can use SageMaker for inference:

1. The first step is to install sagemaker and then import it:

   import sagemaker
   import boto3
   from sagemaker.huggingface import HuggingFaceModel

2. The next step is to run this code to get the role:

   try:
      ...