Deploying the MLOps inner loop
In Databricks, the MLOps inner loop uses a variety of tools within the DI platform that we’ve already touched upon throughout this book, such as MLflow, Feature Engineering with Unity Catalog, and Delta. This chapter will highlight how you can leverage them together to facilitate MLOps from one place. MLOps is covered in even more depth by Databricks’ ebook, The Big Book of MLOps, which we highly recommend if you wish to learn more about the guiding principles and design decisions when architecting your own MLOps solution. We use GitHub to help facilitate DevOps and code reproducibility. For the DataOps portion, we use Unity Catalog and Delta. These tools help us track the versions of data and the code associated with the features created. This is the data reproducibility piece of DataOps. We use Delta time travel to query data from previous versions of the same table in the short term. For long term reproducibility, we recommend saving...
