Section 2 – Tracking a Deep Learning Pipeline at Scale
In this section, we will learn how to use MLflow to track deep learning (DL) pipelines to answer various provenance-tracking questions related to data, model, and code (including both notebook and pipeline code). We will start with setting up a local full-fledged MLflow tracking server that will be used frequently in the rest of this book. A provenance tracking framework that includes six types of provenance questions will be presented to guide our implementation. Then, we will learn how to track model provenance, metrics, and parameters using MLflow to answer these provenance questions. We will also learn how to choose an appropriate notebook and pipeline framework to implement DL code that's extensible and trackable. We will then implement a multi-step DL training/testing/registration pipeline using MLflow's MLproject. Finally, we will learn how to track public and privately built Python libraries and...
