Setting up an ingestion node

The main goals of Elasticsearch are indexing, searching, and analytics, but it's often necessary to modify or enhance the documents before storing them in Elasticsearch.

The following are the most common scenarios in this case: 

• Preprocessing the log string to extract meaningful data
• Enriching the content of textual fields with NLP tools
• Enriching the content using  ML computed fields
• Adding data modification or transformation during ingestion, such as the following:
  • Converting IP in geolocalization
  • Adding DateTime fields at ingestion time
  • Building custom fields (via scripting) at ingestion time

Getting ready

You need a working Elasticsearch installation, as described in the Downloading and installing Elasticsearch recipe, as well as a simple text editor to change configuration files.

How to do it…

To set up an ingest node, you need to edit the config/elasticsearch.yml file and set up the...