Summary

In this chapter, we learned about Apache Spark, its architecture, and Spark ecosystem in brief. Our focus was on covering different ways we can integrate Kafka with Spark and their advantages and disadvantages. We also covered APIs for the receiver-based approach and direct approach. Finally, we covered a small use case about IP fraud detection through the log file and lookup service. You can now create your own Spark streaming application.
In the next chapter, we will cover another real-time streaming application, Apache Heron (successor of Apache Storm). We will cover how Apache Heron is different from Apache Spark and when to use which one.