Elasticsearch, Logstash, and Kibana (ELK stack)

The ELK stack refers to the three software components that collaboratively enrich, index, and visualize data for analysis. Some folks will include the Beats family of data collection agents as part of the ELK stack. We will cover Beats in the next section.

Elasticsearch

Elasticsearch is the central engine of the ELK stack. As we’ll discuss in a moment, Elasticsearch is technically a type of non-relational, NoSQL database. Without the ability to effectively search through the data we have collected, how on Earth would we ever be able to decide if there is something that should be alerted to? Kibana would have no path forward to properly display the visualization threats it provides to us. It is a very powerful component of the ELK stack in that it is designed to handle very large volumes of data in scenarios where a lot of querying frequently occurs.

Elasticsearch accomplishes this by using a technique known as sharding...