Summary
In this chapter, we discussed the workflow management options available in the Databricks environment. We also looked at Workflows with Jobs functionality in more detail in relation to its utility in automating your ML workflows.
We went through a sample workflow of creating a notebook with tests to perform on any new model we want to transition to the Staging stage of the Model Registry. We then configured the Model Registry Jobs webhooks feature to be triggered by another automated model retraining notebook. Similar workflows can make your model tests arbitrarily complex to fit your needs.
In the last chapter, we will cover the concept of model drift and how to trigger a model’s retraining automatically.
