Aggregating logs into Kafka using Log4J

Log management is a critical part of any microservice deployment. When it comes to debugging your application, the two things that matter a lot are logs and metrics. We've already learned how to use metrics to monitor our application, and in this recipe, we will learn how to consolidate our logs. Logs can be stored in plenty of stores. In this recipe, we will look at how to store our logs in a Kafka topic. Once we get our log messages in a Kafka topic, we can use Log Management tools to make some sense out of it.

Getting ready

In this recipe, we will be configuring the geolocation microservice to send log messages over to a Kafka topic called geolocationLogs.

Let's get ready by creating the topic in Kafka. If you don't have Kafka up and running, run it using Docker Compose.

Open a new terminal shell and navigate to the home directory of Kafka. Execute the following command:

        ./bin/kafka-topics.sh --create --topic geolocationLogs ...