Writing a Spark job for Redis

In the previous recipe, we talked about how to write a MapReduce job to read and write data in Redis. With the development of large-scale computing, over the years Apache Spark has gained more popularity than MapReduce. Apache Spark is an open source, big data distributed, computing engine. Compared to MapReduce, it provides better performance as well as more powerful and user-friendly APIs.

When it comes to using Redis in a Spark job, a connector for manipulating data in Redis with Spark is provided by Redis Labs. In this recipe, we'll show you how to use the Spark-Redis connector library to read and write data in Redis.

Getting ready…

You need to finish the installation of the Redis Server as we described in the Downloading and installing Redis recipe in Chapter 1, Getting Started with Redis. You need to use the FLUSHALL command to flush all the data in your Redis instance before moving on to the next section.

The requirements of IDE and JDK are the same as in...