Chapter 3. Collect, Parse and Transform Data with Logstash
By now, we should have a basic understanding of ELK Stack and what role it plays in log analysis or event analysis systems. In Chapter 2, Building Your First Data Pipeline with ELK, we built analytics over the GOOG stock price data with the ELK Stack configuration, and also understood role of each component of the stack in the pipeline.
In this chapter, we will get into more detail on Logstash, the most important component of the ELK Stack, and see how Logstash helps collect, parse, and transform any format and any type of data to a common format, which can be used to build a wide variety of analytics systems across many applications.
We saw in Chapter 1, Introduction to ELK Stack, the importance of log analysis and problem with log analysis methods. Also, there are a variety of log formats, and date and time formats in logs. Often these logs are customized to each application and require expert knowledge to gather essential...
