Implementing and verifying the design

The first step in a real-time implementation like this is to set up the streaming platform. To implement our architecture, we need to install Apache Kafka and create the necessary topics on our local machine.

Setting up Apache Kafka on your local machine

In this section, you will learn how to set up an Apache Kafka cluster, run it, and create and list topics. Follow these steps:

1. Download Apache Kafka version 2.8.1 from https://archive.apache.org/dist/kafka/2.8.1/kafka_2.12-2.8.1.tgz.
2. Extract the kafka_2.12-2.8.1.tgz archive file. The following command will help you do the same on Linux or macOS:

   $ tar -xzf kafka_2.12-2.8.1.tgz
   $ cd kafka_2.12-2.8.1

3. Navigate to the Kafka installation root directory and start zookeeper using the following command:

   $ bin/zookeeper-server-start.sh config/zookeeper.properties

4. Next, run the Kafka server using the following command:

   $ bin/kafka-server-start.sh config/server.properties

5. Next,...