Automating ML Workflows Using Databricks Jobs
In the last chapter, we covered the ML deployment life cycle and the various model deployment paradigms. We also understood how the response latency, the scalability of the solution, and the way we are going to access the predictions play an important role in deciding the deployment method.
In this chapter, we are going to take a look at Databricks Workflows with Jobs (previously called Databricks Jobs). This functionality can be leveraged not only to schedule the retraining of our models at regular intervals but also to trigger tests to check our models when transitioning from one Model Registry stage to another using the webhook integrations we discussed in Chapter 6.
We will be covering the following topics:
- Understanding Databricks Workflows
- Utilizing Databricks Workflows with Jobs to automate model training and testing
