Using dynamic templates in document mapping

In this recipe, we will explore how to leverage dynamic templates in Elasticsearch to automatically apply mapping rules to fields, based on their data types. Elasticsearch allows you to define dynamic templates that simplify the mapping process by dynamically applying mappings to new fields as they are indexed.

Getting ready

Make sure that you have completed the previous recipes:

• Using an analyzer
• Defining index mapping

The snippets of the recipe are available at this address: https://github.com/PacktPublishing/Elastic-Stack-8.x-Cookbook/blob/main/Chapter2/snippets.md#using-dynamic-templates-in-document-mapping.

How to do it…

1. In our example, the default mapping of the year field is set to the long field type, which is suboptimal for storage. We also want to prepare the document mapping so that if additional year fields such as review_year and award_year are introduced, they will have a dynamically applied mapping. Let’s go to Kibana | Dev Tools, where we can extend the previous mapping as follows:

   PUT movies/_mapping
   {
     "dynamic_templates": [{
       "years_as_short": {
         "match_mapping_type": "long",
           "match": "*year",
             "mapping": {
               "type": "short"
             }
       }
     }]
   }

2. Next, we ingest a new document with a review_year field using the following command:

   POST movies/_doc/
   {
     "review_year": 1993,
     "release_year": 1992,
     "title": "Reservoir Dogs",
     "origin": "American",
     "director": "Quentin Tarantino",
     "cast": "Harvey Keitel, Tim Roth, Steve Buscemi, Chris Penn, Michael Madsen, Lawrence Tierney",
     "genre": "crime drama",
     "wiki_page": "https://en.wikipedia.org/wiki/Reservoir_Dogs",
     "plot": "a group of criminals whose planned diamond robbery goes disastrously wrong, leading to intense suspicion and betrayal within their ranks."
   }

3. We can now check the mapping with the following command, and we can see that the movies mapping now contains the dynamic template, and the review_year field correctly maps to short, as shown in Figure 2.16.

   GET /movies/_mapping

Figure 2.16 – Updated mapping for the movies index with a dynamic template

How it works...

In our example for the years_as_short dynamic template, we configured custom mapping as follows:

• The match_mapping_type parameter is used to define the data type to be detected. In our example, we try to define the data type for long values.
• The match parameter is used to define the wildcard for the filename ending with year. It uses a pattern to match the field name. (It is also possible to use the unmatch parameter, which uses one or more patterns to exclude fields matched by match.)
• mapping is used to define the mapping the match field should use. In our example, we map the target field type to short.

There’s more…

Apart from the example that we have seen in this recipe, dynamic templates can also be used in the following scenarios:

• Only with a match_mapping_type parameter that applies to all the fields of a single type, without needing to match the field name
• With patch_match or patch_unmatch for a full dotted patch to the field such as "path_match": "myfield_prefix.*" or "path_unmatch": "*.year".

For timestamped data, it is common to have many numeric fields such as metrics. In such cases, filtering on those fields is rarely required and only aggregation is useful. Therefore, it is recommended to disable indexing on those fields to save disk space. You can find a concrete example in the following documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-templates.html#_time_series.

The default dynamic field mapping in Elasticsearch is convenient to get started, but it is beneficial to consider defining field mappings more strategically to optimize storage, memory, and indexing/search speed. The workflow to design new index mappings can be as follows:

1. Index a sample document containing the desired fields in a dummy index.
2. Retrieve the dynamic mapping created by Elasticsearch.
3. Modify and optimize the mapping definition.
4. Create your index with the custom mapping, either explicit or dynamic.

See also

There are some more resources in Elastic’s official documentation, such as the following:

• Mapping: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html
• Dynamic templates: Templates can also be created by calling REST API with an HTTP client such as CURL/Postman; here is the documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html