Centralized Logging with the EFK Stack
In this chapter, we will learn how to collect and store log records from microservice instances, as well as how to search and analyze log records. As we mentioned in Chapter 1, Introduction to Microservices, it is difficult to get an overview of what is going on in a system landscape of microservices when each microservice instance writes log records to its local filesystem. We need a component that can collect the log records from the microservice's local filesystem and store them in a central database for analysis, search, and visualization. A popular open source-based solution for this is based on the following tools:
- Elasticsearch, a distributed database with great capabilities for searching and analyzing large datasets
- Fluentd, a data collector that can be used to collect log records from various sources, filter and transform the collected information, and finally send it to various consumers, for example, Elasticsearch...
