Setting up Kafka using Docker

In order to demonstrate streaming, we first require a streaming endpoint. A streaming endpoint could be a TCP socket, messaging destination, and so on. In this chapter, we will use Kafka heavily to demonstrate its streaming abilities. In this recipe, we will learn how to set up Kafka using Docker Compose. Before we jump into the recipe and start orchestrating a Kafka instance, let's first take some time to understand how Kafka works.

Kafka

According to Kafka's documentation, it is a distributed streaming platform. It is distributed because it has clustering abilities and is fault tolerant. Achieving fault tolerance is not very easy. But Kafka's architecture makes it fault tolerant and, at the same time, simple to work with and understand. Simplicity is one of the reasons Kafka is being adopted by lot of organizations. At the same time, it is very powerful. It can handle a huge amount of data at once. Kafka utilizes Zookeeper for most of its clustering mechanism...