Split string issue while reading from elasticsearch and logstash db using templating variable

Grafana Elasticsearch
1

We are trying to create a dashboard using elasticsearch. The sample data in the database is shown below:
{
“@timestamp” => 2017-07-04T06:41:22.121Z,
“@version” => “1”,
“tablename” => “RT_TAPIN_TREND”,
“fliter” => “event_type:VOICE and roam_operator:T-Mobile Netherlands BV(prev. Ben Nederland B.V.)”
}

We have a column name called “fliter” in the elasticsearch database. We want to make the selection of string in “fliter” column dynamic, thus we are using templating variable for that. By defining template variable “filter” of type ‘Query’, the entire string is splitted into different words as shown, which is not desired. How to read the entire string as a single value?

Templating_issue.png1366×768 65.9 KB

2

This is due to elasticsearch indexing, change index mapping or change property to a non analyzed (keyword) property.

This must be the single most common ES issue so supprised that a Google search did not give you the answer :slight_smile:

3

Hi Torkel ,
Thank You for the suggestion
I am new to elastic search , logstash and grafana . I tried as per the suggestions in google to change the mapping from analyzed field which is marked in bold to not_analyzed in elasticsearch-templete-ex2x.json.

{
“template” : “logstash-",
“settings” : {
“index.refresh_interval” : “5s”
},
“mappings” : {
“default” : {
"_all" : {“enabled” : true, “omit_norms” : true},
“dynamic_templates” : [ {
“message_field” : {
“path_match” : “message”,
“match_mapping_type” : “string”,
“mapping” : {
“type” : “string”, “index” : “analyzed”, “omit_norms” : true,
“fielddata” : { “format” : “disabled” }
}
}
}, {
“string_fields” : {
“match” : "”,
“match_mapping_type” : “string”,
“mapping” : {
“type” : “string”, “index” : “analyzed”, “omit_norms” : true,
“fielddata” : { “format” : “disabled” },
“fields” : {
“raw” : {“type”: “string”, “index” : “not_analyzed”, “doc_values” : true, “ignore_above” : 256}
}
}
}
}, {
“float_fields” : {
“match” : “",
“match_mapping_type” : “float”,
“mapping” : { “type” : “float”, “doc_values” : true }
}
}, {
“double_fields” : {
“match” : "”,
“match_mapping_type” : “double”,
“mapping” : { “type” : “double”, “doc_values” : true }
}
}, {
“byte_fields” : {
“match” : “",
“match_mapping_type” : “byte”,
“mapping” : { “type” : “byte”, “doc_values” : true }
}
}, {
“short_fields” : {
“match” : "”,
“match_mapping_type” : “short”,
“mapping” : { “type” : “short”, “doc_values” : true }
}
}, {
“integer_fields” : {
“match” : “",
“match_mapping_type” : “integer”,
“mapping” : { “type” : “integer”, “doc_values” : true }
}
}, {
“long_fields” : {
“match” : "”,
“match_mapping_type” : “long”,
“mapping” : { “type” : “long”, “doc_values” : true }
}
}, {
“date_fields” : {
“match” : “",
“match_mapping_type” : “date”,
“mapping” : { “type” : “date”, “doc_values” : true }
}
}, {
“geo_point_fields” : {
“match” : "”,
“match_mapping_type” : “geo_point”,
“mapping” : { “type” : “geo_point”, “doc_values” : true }
}
} ],
“properties” : {
"@timestamp": { “type”: “date”, “doc_values” : true },
"@version": { “type”: “string”, “index”: “not_analyzed”, “doc_values” : true },
“geoip” : {
“type” : “object”,
“dynamic”: true,
“properties” : {
“ip”: { “type”: “ip”, “doc_values” : true },
“location” : { “type” : “geo_point”, “doc_values” : true },
“latitude” : { “type” : “float”, “doc_values” : true },
“longitude” : { “type” : “float”, “doc_values” : true }
}
}
}
}
}
}

But the results are still as splitted strings .Could you please help us in verifying if this is the right file for changes to be made and where exactly we have to make the changes in the above template , does some change need to be made in grafana UI . Appreciate your guidance and help in advance

4

Sorry not a logstash expert. You might have better luck asking in logstash/ES forum

5

Thanks …I could do that by using ES 5.2 and then changing the templating query to {“find”: “terms”, “field”: “fliter.keyword”}