ElasticSearch index template versioning

elasticsearch index template dynamic mapping
elasticsearch index template example
elasticsearch index template tutorial
elasticsearch delete template
elasticsearch template index_patterns
elasticsearch search template
elasticsearch template vs mapping
elasticsearch index mapping

My application creates an index template on ElasticSearch. Later on, the application is updated and needs to update the index template to support new fields.

  1. Is there a way to set a some identifier for the index template (except its name which is always the same), such that my app can compare these values between the app version and the deployed version? (unique ID / ETag / anything else)

  2. Alternatively, if I can't detect a difference: What is the cost of modifying the index template multiple times, mostly modifications that perform no actual change?

  3. Can I find for an index which template it uses (and which version of it, if such ability exists)?

Thanks.


  1. Yes, you can use ElasticSearch's template versioning feature by including "version": <version number> in your template.

Source: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html#versioning-templates

  1. There's no cost in modifying the template, as it's just stored in ES's database {read as 'storage used by ES'}. The only difference is that updates to the templates will only be applicable to the new index being created and won't be applied to the existing indices.

Index templates | Elasticsearch Reference [7.8], Templates are configured prior to index creation and then when an index is created either manually or through indexing a document, the template settings are  An index template is a way to tell Elasticsearch how to configure an index when it is created. Templates are configured prior to index creation and then when an index is created either manually or through indexing a document, the template settings are used as a basis for creating the index.


When creating an index template, you can use the mapping metadata to store the extra info needed to know from which version of the application the mapping was created.

PUT _template/my_template
{
  "template" : "logstash-*",
  "mappings" : {
      "logs" : {
        "_meta" : {
          "product" : "my_product",
          "version" : "1.0.1"
        },
        "_source" : {"enabled" : "true"},
        "properties" : {
        }
      }
  }
}

Then query for existing indices to find the index last created and then get its mappings.

GET logstash-2015.10.08/_mapping

You'll find the mappings and the metadata inside.

This can be useful to store the version of the product the mapping correlates to.

Index Templates | Elasticsearch Reference [6.8], Template Versioningedit. Templates can optionally add a version number, which can be any integer value, in order to simplify template management by external  Index Templates edit Index templates allow you to define templates that will automatically be applied when new indices are created. The templates include both settings and mappings and a simple pattern template that controls whether the template should be applied to the new index. Templates are only applied at index creation time.


I've solved for this before by using version numbers, build IDs or time stamps as part of the index name (and using index aliases to point to the relevant index for each application).

See this post for a more detailed example: https://www.elastic.co/blog/changing-mapping-with-zero-downtime

ES Template versioning - Elasticsearch, a question about the use case of version field in the index templates. /indices​-templates.html#versioning-templates I have observed that I… If the index name matches more than one template, Elasticsearch merges all mappings and settings from all matching templates and applies them to the index. The settings from the more recently created index templates override the settings of older index templates.


Put index template API | Elasticsearch Reference [7.8], Multiple matching templates with the same order value will result in a non-​deterministic merging order. Template versioningedit. You can use the version  To use a different name, set the index option in the Elasticsearch output. The value that you specify should include the root name of the index plus version and date information. You also need to configure the setup.template.name and setup.template.pattern options to match the new name. For example:


Index templates | Elasticsearch Reference [master], Templates are configured prior to index creation and then when an index is created either manually or through indexing a document, the template settings are  In Elasticsearch, index templates are used to define settings and mappings that determine how fields should be analyzed. The recommended index template file for Heartbeat is installed by the Heartbeat packages.


Elasticsearch Versioning Support, Template versioningedit. You can use the version parameter to add an optional version number to an index template. External systems can use these version  Adds a JSON document to the specified index and makes it searchable. If the document already exists, updates the document and increments its version. (Required, string) Name of the target index. By default, the index is created automatically if it doesn’t exist. For more information, see Create