Elasticsearch Mustache optional parameters

mustache language elasticsearch
elasticsearch parameterized query
elasticsearch search template performance
search by template
mustache if
mustache escape
mustache hash
elasticsearch custom index

I've been struggling with Elasticsearch templates, specifically in optional parameters. I'd like to add optional filters there. This is the code snippet I was trying out:

{
  "filter" : {
    "bool" : {
      "must" : [
        {{#ProductIDs.0}}
        { "terms" : { "Product.ProductID" : [{{#ProductIDs}}{{.}},{{/ProductIDs}}] } }
        {{/ProductIDs.0}}
      ]
    }
  }
}

Of course I replaced " with \", uglified it, wrapped it up in { "template" :"_snippet_above_" }.

Now when I'm trying to call it using the following:

GET /statistic/_search/template
{
    "template": {
        "id": "test" 
    },
    "params": {
        "ProductIDs": [1,2]
    }
}

It ignores parameter that I've provided, however when I try to do that in official mustache.io demo page - it works just fine.

I tried {{#ProductIDs.length}} option too - it didn't work out. After doing some research I've found out that there is one difference between mustache.js and mustache.java. I assumed that Elasticsearch uses JAVA version and it doesn't support length parameter, so I have to rely on isEmpty. So I've rewritten my query as follows:

{
  "filter" : {
    "bool" : {
      "must" : [
        {{^ProductIDs.isEmpty}}
        { "terms" : { "Product.ProductID" : [{{#ProductIDs}}{{.}},{{/ProductIDs}}] } }
        {{/ProductIDs.isEmpty}}
      ]
    }
  }
}

Now when I query template with ProductIDs list - it works fine, however if I remove parameter, it brings no results. I assume it generates this:

{
  "filter" : {
    "bool" : {
      "must" : [
        { "terms" : { "Product.ProductID" : [] } }
      ]
    }
  }
}

If I send empty array as Parameter - it works fine.

GET /statistic/_search/template
{
    "template": {
        "id": "test" 
    },
    "params": {
        "ProductIDs": []
    }
}

I assume this happens because "ProductIDs" are undefined and not empty.

Is there a way to cath this condition in mustache.java so I can ignore these parameters?

tl;dr; The issue is that if I don't specify parameter in my search request via template, my condition is rendered as an empty array, see this:

{
  "filter" : {
    "bool" : {
      "must" : [
        { "terms" : { "Product.ProductID" : [] } }
      ]
    }
  }
}

If I pass empty array as a parameter, see this:

GET /statistic/_search/template
{
    "template": {
        "id": "test" 
    },
    "params": {
        "ProductIDs": []
    }
}

It works as expected and doesn't generate filter condition as described in my template, because array doesn't have any data in it.

I want this:

GET /statistic/_search/template
{
    "template": {
        "id": "test" 
    },
    "params": {
    }
}

To work same as this:

GET /statistic/_search/template
{
    "template": {
        "id": "test" 
    },
    "params": {
        "ProductIDs": []
    }
}

A workaround probably not the most elegant would be to change template query to be a should clause and add a match_all clause for empty list.

example:

{
    "filter" : { 
        "bool" : {
            "should" : [  
                { "terms" : { "status" : [ "{{#ProductIDs}}","{{.}}","{{/ProductIDs}}"] }} 
                {{^ProductIDs}},
                {"match_all":{}}
                {{/ProductIDs}}
            ]
        }
    }
}

Search Template | Elasticsearch Reference [7.8], The /_search/template endpoint allows you to use the mustache language to pre- render search The line_no , start , and end parameters are optional. Elasticsearch Mustache optional parameters. Ask Question Asked 4 years, 7 months ago. Active 5 months ago. Viewed 1k times 3. I've been struggling with Elasticsearch

Didn't try it, but shouldn't something like this work?

{
  "filter" : {
    "bool" : {
      "must" : [
        {{#ProductIDs}}
          {{^ProductIDs.isEmpty}}
            { "terms" : { "Product.ProductID" : [{{#ProductIDs}}{{.}},{{/ProductIDs}}] } }
          {{/ProductIDs.isEmpty}}
          {{#ProductIDs.isEmpty}}
            {"match_all":{}}
          {{/ProductIDs.isEmpty}}
        {{/ProductIDs}}
        {{^ProductIDs}}
          {"match_all":{}}
        {{/ProductIDs}}
      ]
    }
  }
}

Ain't pretty, maybe there's better way.

Search Template | Elasticsearch Reference [6.8], The /_search/template endpoint allows to use the mustache language to pre render search requests, The line_no , start , and end parameters are optional. (Optional, boolean) If true, hits.total are rendered as an integer in the response. Defaults to false. routing (Optional, string) Target the specified primary shard. scroll (Optional, time units) Specifies how long a consistent view of the index should be maintained for scrolled search. search_type (Optional, string) The type of the search operation.

My suggestion to overcome this using a JSON template is:

{
   "query": {
      "bool": {
        "must": [
            {
                "script": {
                    "script": {
                        "inline": "1==1 {{#ProductIDs}} &&  [\"{{#ProductIDs}}\",\"{{.}}\",\"{{/ProductIDs}}\"].contains(doc['Product.ProductID'].value){{/ProductIDs}}",
                        "lang": "painless"
                    }
                }
            }
        ]
    }
}

Clean up your Elasticsearch query logic with search templates , I stumbled onto Elasticsearch's search templates feature on my last project That example specified the template and the parameters in the same request. in $ES_HOME/config/scripts and is named template-id.mustache. Example. By default, the full indexed document is returned as part of all searches. This is referred to as the source (_source field in the search hits).If we don’t want the entire source document returned, we have the ability to request only a few fields from within source to be returned, or we can set _source to false to omit the field entirely.

Search templates (Stored Procedures in Elasticsearch) – Abhishek , The template accepts parameters, which can be specified at the runtime. Search Templates are expressed using the Mustache template engine. The above query with the optional variation parameter is shown here:. Elasticsearch documentation is nice and detailed on this topic. I was confident that it would not be too hard to find resources on this as the use case seemed like one that a lot of AWS based projects would come across. We are wanting to create an Elasticsearch cluster where each node is housed in its own EC2.

Search Templates, Variables are represented with double braces in Mustache notation. If the parameter is not defined in the params section, Elasticsearch uses the default value. (Optional, time value) Period to retain the search context for scrolling. See Scroll. This value overrides the duration set by the original search API request’s scroll parameter. By default, this value cannot must be less than 1d (one day). You can change this limit using the search.max_keep_alive cluster-level setting.

Logic-less templates., By default a variable "miss" returns an empty string. This can usually be configured in your Mustache library. The Ruby version of Mustache supports raising an  An optional transform to transform the payload before executing the webhook action. An optional throttle period for the action (5 minutes in this example) The HTTP method to use when connecting to the host. The host to connect to. The port to connect to. The path (URI) to use in the HTTP request. The body to send with the request

Comments
  • I don't understand what the problem is and what is the expectation.
  • Sorry for the confusion, @AndreiStefan I've updated question. Let me know if this makes sense now.
  • Well, this is workaround I thought about. But I am interested in the actual solution. If I not pass any parameter - not to render it at all. :(
  • given that mustache.java has no support for accessing array elements i don't think there is a way to get around this but to restructure your template query as above or use an additional param that has the length of list and use that as the conditional to introduce the must clause
  • I'm kinda only interested in checking if given parameter exists or is not empty. Seems that passing an empty array/list is only solution, no?
  • yeah true that works too however if you want a result when no ProductsID is passed you would require the workaround of changing the query or using a flag param