Illegal_argument_exception when querying Elastic Search Cloud Deployment

Hi, I’m getting this error when querying elasticsearch cloud deployment from grafana cloud (I’m copying the whole trace at the end of the post):

type:“illegal_argument_exception”
reason:“field expansion matches too many fields, limit: 1024, got: 1433”

There is a workaround that recommends updating some setting on the database (https://github.com/elastic/kibana/issues/15863), but unfortunately this setting cannot be changed on cloud deployment.

What surprises me is that I can make the same query against the same index (with no setting updated on the database) from Kibana. So I guess that some parametrization might be done from the dashboarding application to overcome this issue.

Any idea?

Thanks a lot!

{
“xhrStatus”: “complete”,
“request”: {
“method”: “POST”,
“url”: “api/datasources/proxy/3/_msearch?max_concurrent_shard_requests=5”,
“data”: “{“search_type”:“query_then_fetch”,“ignore_unavailable”:true,“index”:“twitter_marvel_elastic_example”}\n{“size”:0,“query”:{“bool”:{“filter”:[{“range”:{”@timestamp":{“gte”:“1572868208628”,“lte”:“1572871808628”,“format”:“epoch_millis”}}},{“query_string”:{“analyze_wildcard”:true,“query”:""}}]}},“aggs”:{“2”:{“date_histogram”:{“interval”:“5s”,“field”:"@timestamp",“min_doc_count”:0,“extended_bounds”:{“min”:“1572868208628”,“max”:“1572871808628”},“format”:“epoch_millis”},“aggs”:{}}}}\n"
},
“response”: {
“took”: 12,
“responses”: [
{
“error”: {
“root_cause”: [
{
“type”: “query_shard_exception”,
“reason”: "failed to create query: {\n “bool” : {\n “filter” : [\n {\n “range” : {\n “@timestamp” : {\n “from” : “1572868208628”,\n “to” : “1572871808628”,\n “include_lower” : true,\n “include_upper” : true,\n “format” : “epoch_millis”,\n “boost” : 1.0\n }\n }\n },\n {\n “query_string” : {\n “query” : "
",\n “fields” : [ ],\n “type” : “best_fields”,\n “default_operator” : “or”,\n “max_determinized_states” : 10000,\n “enable_position_increments” : true,\n “fuzziness” : “AUTO”,\n “fuzzy_prefix_length” : 0,\n “fuzzy_max_expansions” : 50,\n “phrase_slop” : 0,\n “analyze_wildcard” : true,\n “escape” : false,\n “auto_generate_synonyms_phrase_query” : true,\n “fuzzy_transpositions” : true,\n “boost” : 1.0\n }\n }\n ],\n “adjust_pure_negative” : true,\n “boost” : 1.0\n }\n}",
“index_uuid”: “kE3y3OR1QZ-Pbr7DYKIdng”,
“index”: “twitter_marvel_elastic_example”
}
],
“type”: “search_phase_execution_exception”,
“reason”: “all shards failed”,
“phase”: “query”,
“grouped”: true,
“failed_shards”: [
{
“shard”: 0,
“index”: “twitter_marvel_elastic_example”,
“node”: “vLhmgrNFSDqITyigl3JhcQ”,
“reason”: {
“type”: “query_shard_exception”,
“reason”: “failed to create query: {\n “bool” : {\n “filter” : [\n {\n “range” : {\n “@timestamp” : {\n “from” : “1572868208628”,\n “to” : “1572871808628”,\n “include_lower” : true,\n “include_upper” : true,\n “format” : “epoch_millis”,\n “boost” : 1.0\n }\n }\n },\n {\n “query_string” : {\n “query” : “",\n “fields” : [ ],\n “type” : “best_fields”,\n “default_operator” : “or”,\n “max_determinized_states” : 10000,\n “enable_position_increments” : true,\n “fuzziness” : “AUTO”,\n “fuzzy_prefix_length” : 0,\n “fuzzy_max_expansions” : 50,\n “phrase_slop” : 0,\n “analyze_wildcard” : true,\n “escape” : false,\n “auto_generate_synonyms_phrase_query” : true,\n “fuzzy_transpositions” : true,\n “boost” : 1.0\n }\n }\n ],\n “adjust_pure_negative” : true,\n “boost” : 1.0\n }\n}",
“index_uuid”: “kE3y3OR1QZ-Pbr7DYKIdng”,
“index”: “twitter_marvel_elastic_example”,
“caused_by”: {
“type”: “illegal_argument_exception”,
“reason”: “field expansion matches too many fields, limit: 1024, got: 1433”
}
}
}
]
},
“status”: 400
}
],
“$$config”: {
“method”: “POST”,
“url”: “api/datasources/proxy/3/_msearch?max_concurrent_shard_requests=5”,
“data”: “{“search_type”:“query_then_fetch”,“ignore_unavailable”:true,“index”:“twitter_marvel_elastic_example”}\n{“size”:0,“query”:{“bool”:{“filter”:[{“range”:{”@timestamp":{“gte”:“1572868208628”,“lte”:“1572871808628”,“format”:“epoch_millis”}}},{“query_string”:{“analyze_wildcard”:true,“query”:"
”}}]}},“aggs”:{“2”:{“date_histogram”:{“interval”:“5s”,“field”:”@timestamp",“min_doc_count”:0,“extended_bounds”:{“min”:“1572868208628”,“max”:“1572871808628”},“format”:“epoch_millis”},“aggs”:{}}}}\n"
}
}
}

Did you get a solution to this? I am facing same issue on Elastic 7.7, Kibana works fine whereas Grafana doesnt. Tried changing indices.query.bool.max_clause_count to a higher value in ES yml file, but still getting same error.