Data Parsing and Transformation
The first phases of the data journey is the input phase, which we discussed in detail in Chapter 9, Configuring Splunk Data Inputs. Data parsing is the second phase, followed by data being indexed on the disk. This chapter deals with the parsing phase, which comes right after the input phase and ends by handing over the data to the index phase for storage and preparation for data searching.
The question that might arise is what the need for the parsing phase is, as all the data has been collected, the metadata fields are set during the input phase, and finally, data is forwarded to indexers for indexing. The prominent features of the parsing phase are breaking the whole data stream into individual events, extracting and applying timestamps, setting the metadata fields to individual events, manipulating metadata before indexing, and transforming the data if needed. During the input phase, metadata fields such as the index, host, sourcetype, source...
