Chapter 12: Using the Ingest Module

Elasticsearch 8.x introduces a set of powerful functionalities that target the problems that arise during the ingestion of documents via the ingest node.

In Chapter 1, Getting Started, we discussed that the Elasticsearch node can have different roles and the main important ones are master, data, and ingest; the idea of splitting the ingest component from the others is to create a more stable cluster due to problems that can arise when preprocessing documents (mainly due to the custom plugin used in the ingest part, which could require restarting the ingest nodes to be updated).

To create a more stable cluster, the ingest nodes should be isolated by the master nodes (and possibly also from the data ones) in case some problems occur, such as a crash due to plugins such as the attachment plugin and high loads due to complex type manipulation.

The ingestion node can replace a Logstash installation in simple scenarios.

In this chapter, we...