10. Distributed machine learning on Azure

In the previous chapter, we learned about hyperparameter tuning, through search and optimization using HyperDrive as well as Azure Automated Machine Learning, as a special case of hyperparameter optimization, involving feature engineering, model selection, and model stacking. Automated machine learning is machine learning as a service (MLaaS) where the only input is your data, a ML task, and an error metric. It's hard to imagine running all the experiments and parameter combinations for Azure Automated Machine Learning on a single machine or a single CPU/GPU—we are looking into ways to speed up the training process through parallelization and distributed computing.

In this chapter, we will take a look into distributed and parallel computing algorithms and frameworks for efficiently training ML models in parallel. The goal of this chapter is to build an environment in Azure where you can speed up the training process of...