Data preparation

As Kibana is all about gaining insight from data, let's load some sample data that we will use as we follow the tutorial. One of the most common use cases is log analysis. For this tutorial, we will be loading Apache server logs into Elasticsearch using Logstash and then using it in Kibana for analysis/building visualizations.

https://github.com/elastic/elk-index-size-tests hosts a dump of Apache server logs that were collected for the site www.logstash.net for the period of May 2014 to June 2014. It contains 300,000 log events.

Navigate to https://github.com/elastic/elk-index-size-tests/blob/master/logs.gz and click the Download button. Unzip the logs.gz file.

Make sure you have Logstash version 5.6 and above installed. Create a config file named apache.conf in the $LOGSTASH_HOME\bin folder, as shown in the following code block:

input 
{ 
  file {
        path => "D:\Learnings\data\logs\logs"
        type => "logs"
        start_position => "beginning"
       }
}
...