Common RDD transformations

As described in Chapter 1, Introduction of Spark, if an operation on an RDD outputs an RDD then it is a transformation. In other words, transformation on an RDD executes a function on the elements of the RDD and creates a new RDD, which represents the transformed elements. The number of elements in the target RDD can be different than the source RDD.

Transformations in Spark are lazy operations that get triggered with an action; that is, a chain of transformations will only execute when an action is performed.

One important aspect of transformation also deals with the dependency on partition of the parent RDD. If each partition of the generated RDD depends on a fixed partition of the parent RDD then it's called a narrow dependency. Similarly, a wide dependency pertains to operations where the generated RDD depends on multiple partitions of the parent RDD. Typically, map, filter, and similar operations (which will be discussed in the coming section) have a narrow...