In this chapter, we covered the basics of ELK Stack and their characteristics. We explained how we can use Beats to send logs data, file data, and system metrics to Logstash or Elasticsearch and that Logstash can be configured as a pipeline to modify the data format and then send the output to Elasticsearch. Elasticsearch is a search engine built on top of Lucene. It can store data and provide functionality to do full text searching on data. Kibana can be configured to read Elasticsearch data and create visualizations and dashboards. We can embed these dashboards on existing web pages, which can then be used for decision-making.
Then, we discussed different use cases of ELK Stack. The first one we mentioned was log management, which is the primary use case of ELK Stack and which made it famous. In log management, we can capture logs from different servers/sources and dump them in a central Elasticsearch cluster after modifying it through Logstash. Kibana is used to create meaningful graphical visualization and dashboards by reading the Elasticsearch data. Finally, we discussed security monitoring and alerting, where ELK Stack can be quite helpful. Security is a very important aspect of any software, and often it is the most neglected part of development and monitoring. Using ELK Stack, we can observe any security threat.
