Transformations

Transformations on DStreams are similar to those that are applicable to a Spark Core RDD. DStreams consist of RDDs, so a transformation applies to each RDD to generate a transformed RDD for each RDD, creating a transformed DStream. Each transformation creates a specified DStream derived class.

There are many DStream classes that are built for a functionalities; map transformations, window functions, reduce actions, and different InputStream types are implemented using different DStream-derived classes.

The following table showcases the possible  types of transformations:

Transformation	Meaning
map(func)	Applies the transformation function to each element of the DStream and returns a new DStream.
filter(func)	Filters out the records of the DStream to return a new DStream.
repartition(numPartitions)	Creates more or fewer partitions to redistribute the data to change the parallelism.
union(otherStream)	Combines the elements in two source DStreams and returns a new DStream.
count()	Returns...