Orchestrating an ML pipeline with SageMaker Pipelines

The template we're using contains an ML lifecycle pipeline that carries out data preprocessing, data quality checks, model training, model evaluation steps, and eventually model registration. This pipeline is a central piece of the MLOps process where the model is being created. The pipeline is defined in <project-name-prefix>-modelbuild using SageMaker Pipelines. SageMaker Pipelines is an orchestration tool for ML workflow in SageMaker. SageMaker Pipelines integrates with SageMaker Processing, training, Experiments, hosting, and the model registry. It provides reproducibility, repeatability, and tracks data/model lineage for auditability. Most importantly, you can visualize the workflow graph and runtime live in SageMaker Studio. The pipeline can be found under the Pipelines tab in the details portal as shown in Figure 11.8.

Figure 11.8 – A list of pipelines in the project

Note

I have...