Using index components and templates

Real-world index mapping can be very complex and often, parts of it can be reused between different indices types. To be able to simplify this management, mappings can be divided into the following:

• Components: These will collect the reusable parts of the mapping.
• Index templates: These aggregate the components in a single template.

Using components is the most manageable way to scale on large index mappings because they can simplify large template management.

Getting ready

You will need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe of Chapter 1, Getting Started.

To execute the commands in this recipe, you can use any HTTP client, such as curl (https://curl.haxx.se/), Postman (https://www.getpostman.com/), or similar. I suggest using the Kibana console, which provides code completion and better character escaping for Elasticsearch.

How to do it…

We want to build an index mapping composed of two reusable components. To achieve this, follow these steps:

1. First, we will create three components for the timestamp, order, and items. These will store parts of our index mapping:

   PUT _component_template/timestamp-management
   { "template": {
       "mappings": {
         "properties": {
           "@timestamp": { "type": "date"  } } } } }
   PUT _component_template/order-data
   { "template": {
       "mappings": {
         "properties": {
           "id": { "type": "keyword" },
           "date": { "type": "date" },
           "customer_id": { "type": "keyword" },
           "sent": { "type": "boolean" } } } } }
   PUT _component_template/items-data
   { "template": {
       "mappings": {
         "properties": {
           "item": {
             "type": "object",
             "properties": {
               "name": { "type": "keyword" },
               "quantity": { "type": "long" },
               "cost": { "type": "alias", "path": "item.price" },
               "price": { "type": "double" },
               "vat": { "type": "double" } } } } } } }

2. Now, we can create an index template that can sum them up:

   PUT _index_template/order
   {
     "index_patterns": ["order*"],
     "template": {
       "settings": { "number_of_shards": 1 },
       "mappings": {
         "properties": { "id": { "type": "keyword" } } 
        },
       "aliases": { "order": { } }
     },
     "priority": 200,
     "composed_of": ["timestamp-management", "order-data", "items-data"],
     "version": 1,
     "_meta": { "description": "My order index template" } }

How it works…

The process of using index components to build indices templates is very simple: you can register as many components as you wish (Steps 1 and 2 in this recipe) and then aggregate them when you define the template (Step 3). By using this approach, your template is divided into blocks, and the index template is simpler to manage and easily reusable.

For simple use cases, using components to build indices template is too verbose. This approach shines when you need to manage different logs or documents in Elasticsearch that have common parts because you can refactorize them very quickly and reuse them.

Components are simple partial templates that are merged in an index template. Here, the parameters are as follows:

• index_patterns: This is a list of index glob patterns. When an index is created, if its name matches the glob patterns, the template is applied when the index is created.
• aliases: This is an optional alias definition to be applied to the created index.
• template: This is the template to be applied to the index.
• priority: This is an optional order of priority for applying this template. The standard priority of ELK components is 100, so if the value is set below 100, a custom template can override an ELK one.
• version: This is an optional incremental number that is managed by the user to keep track of the updates that are made to the template.
• _meta: This is an optional JSON object that contains metadata for the index.
• composed_of: This is an optional list of index components that are merged to build the final index mapping.

  Note

  This functionality is available from Elasticsearch version 7.8 and above.

See also

The Adding metadata to a mapping recipe in this chapter about using the _meta field.