Data organization

Before we can effectively wrangle all of this underlying data, we need to smartly segment it, possibly enrich it, and leverage the contextual information contained within it. First, we will focus on segmentation and enrichment.

Effective data segmentation

Simply by virtue of collecting some types of data (system performance metrics, log files, and so on) from underlying servers/hosts, there is likely already a natural segmentation of the data by server/host. Let's look at a sample measurement from Metricbeat:

{  
   "_index":"metricbeat-6.0.0-2018.01.01",
   "_type":"doc",
   "_id":"ZQtas2ABB_sNnq-vMrgR",
   "_score":1,
   "_source...