Summary
In this chapter, we introduced machine learning applications. We covered one of the most important topics in machine learning, called feature engineering. Additionally, we provided code examples using Spark ML APIs to build a classification pipeline and a clustering application. Additionally, we also introduced a few tools and utilities that can help select features and build models more easily and efficiently.
In the next chapter, we will introduce GraphFrame applications and provide examples of using Spark SQL DataFrame/Dataset APIs to build graph applications. We will also apply various graph algorithms to graph applications.