Additional open source tools and frameworks –Spark MLlib, Ray, and PyTorch on Google Cloud

In this section, I’ll introduce additional open source tools and frameworks, such as PyTorch, Ray, and Spark Machine Learning Library (MLlib), and demonstrate how they can be used to implement AI/ML workloads on Google Cloud.

Spark MLlib

We introduced Apache Spark in previous chapters and used it for data processing to perform feature engineering in Chapter 6. Apache Spark MLlib is a component of Apache Spark that provides ML tools and algorithms that are optimized for parallel processing with large datasets. In addition to feature engineering, we can use the tools in MLlib to implement various stages in our ML model development life cycle, such as model training, model evaluation, hyperparameter tuning, and prediction, as well as assembling stages into a pipeline that can be executed end to end.

Just as we discussed in the context of data processing, one of the main advantages...