Summary

In this chapter, we learned about some of the valuable features of Azure Databricks that allow us to track training runs, as well as find the optimal set of hyperparameters of machine learning models, using the MLflow Model Registry. We have also learned how we can optimize how we scan the search space of optimal parameters using Hyperopt. This is a great set of tools because we can fine-tune models that have complete tracking for the hyperparameters that are used for training. We also explored a defined search space of hyperparameters using adaptative search strategies, which are much more optimized than the common grid and random search strategies.

In the next chapter, we will explore how to use the MLflow Model Registry, which is integrated into Azure Databricks. MLflow makes it easier to keep track of the entire life cycle of a machine learning model and all the associated parameters and artifacts used in the training process, but it also allows us to deploy these models...