Application optimizations

As discussed, there are multiple ways to improve the performance of your Spark applications. In the previous section, we covered some optimizations that were related to hardware. In this section, we'll discuss how you can apply some optimizations while writing your Spark applications.

Language choice

One of the first choices the developers have to make is to decide the language API they are going to write their applications in. In Chapter 1, Introduction to Apache Spark, we gave an overview of all the languages supported by Spark. The choice of language depends on the use case and the dynamics of the team. If you are part of a data science team and comfortable in writing your machine-learning...