Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Apache Spark 2.x for Java Developers

You're reading from   Apache Spark 2.x for Java Developers Explore big data at scale using Apache Spark 2.x Java APIs

Arrow left icon
Product type Paperback
Published in Jul 2017
Publisher Packt
ISBN-13 9781787126497
Length 350 pages
Edition 1st Edition
Languages
Concepts
Arrow right icon
Authors (2):
Arrow left icon
Sourav Gulati Sourav Gulati
Author Profile Icon Sourav Gulati
Sourav Gulati
Sumit Kumar Sumit Kumar
Author Profile Icon Sumit Kumar
Sumit Kumar
Arrow right icon
View More author details
Toc

Table of Contents (12) Chapters Close

Preface 1. Introduction to Spark FREE CHAPTER 2. Revisiting Java 3. Let Us Spark 4. Understanding the Spark Programming Model 5. Working with Data and Storage 6. Spark on Cluster 7. Spark Programming Model - Advanced 8. Working with Spark SQL 9. Near Real-Time Processing with Spark Streaming 10. Machine Learning Analytics with Spark MLlib 11. Learning Spark GraphX

Common RDD transformations

As described in Chapter 1, Introduction of Spark, if an operation on an RDD outputs an RDD then it is a transformation. In other words, transformation on an RDD executes a function on the elements of the RDD and creates a new RDD, which represents the transformed elements. The number of elements in the target RDD can be different than the source RDD.

Transformations in Spark are lazy operations that get triggered with an action; that is, a chain of transformations will only execute when an action is performed.

One important aspect of transformation also deals with the dependency on partition of the parent RDD. If each partition of the generated RDD depends on a fixed partition of the parent RDD then it's called a narrow dependency. Similarly, a wide dependency pertains to operations where the generated RDD depends on multiple partitions of the parent...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $19.99/month. Cancel anytime