Passing functions to Spark (Java)
In Java, to create a function you will have to implement the interfaces available in the org.apache.spark.api.java function package. There are two popular ways to create such functions:
- Implement the interface in your own class, and pass the instance to Spark.
- Starting Java 8, you can use Lambda expressions to pass off the functions to the Spark framework.
Let's implement the preceding word count examples in Java:
Figure 2.13: Code example of Java implementation of word count (inline functions)
If you belong to a group of programmers who feel that writing inline functions makes the code complex and unreadable (a lot of people do agree to that assertion), you may want to create separate functions and call them as follows:
Figure 2.14: Code example of Java implementation of word count
