Managing ML workflows with AWS Step Functions and the Data Science SDK

AWS Step Functions is a serverless orchestration service that helps integrate and sequence tasks using multiple AWS services. With this service, we just need to focus on configuring the workflows and worry less about the operational overhead of managing distributed and complex applications.

In this recipe, we will use the Data Science SDK to create and manage automated ML workflows with AWS Step Functions. We will build on top of the recipes from Chapter 1, Getting Started with Machine Learning Using Amazon SageMaker, where we trained and deployed a linear learner model to solve a regression problem. Once we have completed the steps in this recipe, we will be able to execute an end-to-end automated workflow using Step Functions state machines, without having to run scripts manually inside Jupyter notebooks.

Getting ready

Here are the prerequisites for this recipe:

• You will need a SageMaker Studio...