Writing a streaming program using Apache Spark

You might be wondering what Apache Spark has to do with microservices. The answer is pretty simple: streaming and data processing. Not all microservices will require streaming, but most of them these days do.

There are two ways you can feed data to a microservice: via REST or message brokers. With RESTful APIs, you can achieve the performance you expect. But it has its own limitations, which is the reason companies move towards message brokers such as Kafka, RabbitMQ, and ZeroMQ. Using frameworks such as Kafka, you can achieve tremendous performance and live results. In fact, today's streaming is all about live results. Before we jump into the recipe, let's take a minute to understand Spark and some of its concepts.

Apache Spark is a fast data-processing framework. It has four major modules: Spark Streaming, Spark SQL, Spark MLlib, and Spark Graphx. Spark Streaming is used to stream data from messaging endpoints such as TCP Socket and Kafka....