Parallelization with Spark RDDs

Now that we know how to create RDDs within the text file that we received from the internet, we can look at a different way to create this RDD. Let's discuss parallelization with our Spark RDDs.

In this section, we will cover the following topics:

• What is parallelization?
• How do we parallelize Spark RDDs?

Let's start with parallelization.

What is parallelization?

The best way to understand Spark, or any language, is to look at the documentation. If we look at Spark's documentation, it clearly states that, for the textFile function that we used last time, it reads the text file from HDFS.

On the other hand, if we look at the definition of parallelize, we can see that this is...