Elastic Query Terms Size No Limit = 500

I’m trying to do a basic elasticsearch query that has a count metric and a group by terms then by geohash grid. It all runs as expected except that the aggregation is limited to a size of 500 and what I need is something around 2,000. The drop down for the “Size” option on the group by line has a “No Limit” choice, but that only codes a size of 500 into the query. Is there any way to increase it to some arbitrary new limit? It sort of lets you do a write in but then just clears it to the “No Limit” option again.

BTW, normally if this was a histogram you would never need >500 groups. However in my application I’m plotting geohash bubbles on a world map. I’m also using a variable in the query and had considered hardcoding a query into a raw document option but variables seem not to be allowed in that scenario.

Any suggestions? Thanks,

John…

Hi,

According to issue this should be fixed from v4.2. However when I test in the latest master I cannot set a custom size which was implied in referenced issue.

What version are you using?

Marcus

Yup, that’s my issue. Currently I am using v5.0.0-beta4. However, prior to that I was on v4.6.3 and believe the issue was still present there as well.

John…

I’ve created an issue for this here.

You’re more than welcome to submit a pull request for this if you want. We love contributions.

Marcus

This still appears to be happening in 5.4

I am trying the following query

and if I set the size to “No Limit” and look in the query inspector the size attribute in the request is set to 500

 {"search_type":"query_then_fetch","ignore_unavailable":true,"index":"logs*","max_concurrent_shard_requests":256} {"size":0,"query":{"bool":{"filter":[{"range":{"@timestamp":{"gte":"1552048546355","lte":"1552070146355","format":"epoch_millis"}}},{"query_string":{"analyze_wildcard":true,"query":"datacenter: achm*"}}]}},"aggs":{"4":{"terms":{"field":"datacenter.keyword","size":500,"order":{"_count":"desc"},"min_doc_count":1},"aggs":{"3":{"terms":{"field":"http_status_code","size":500,"order":{"_key":"desc"},"min_doc_count":1},"aggs":{"2":{"date_histogram":{"interval":"1m","field":"@timestamp","min_doc_count":0,"extended_bounds":{"min":"1552048546355","max":"1552070146355"},"format":"epoch_millis"},"aggs":{}}}}}}}} "

if I then set size to 1000 I get the following:

{"search_type":"query_then_fetch","ignore_unavailable":true,"index":"logs*","max_concurrent_shard_requests":256} {"size":0,"query":{"bool":{"filter":[{"range":{"@timestamp":{"gte":"1552048355483","lte":"1552069955483","format":"epoch_millis"}}},{"query_string":{"analyze_wildcard":true,"query":"datacenter: achm*"}}]}},"aggs":{"4":{"terms":{"field":"datacenter.keyword","size":1000,"order":{"_count":"desc"},"min_doc_count":1},"aggs":{"3":{"terms":{"field":"http_status_code","size":1000,"order":{"_key":"desc"},"min_doc_count":1},"aggs":{"2":{"date_histogram":{"interval":"1m","field":"@timestamp","min_doc_count":0,"extended_bounds":{"min":"1552048355483","max":"1552069955483"},"format":"epoch_millis"},"aggs":{}}}}}}}} "

It appears that No Limit means 500 ? How do I make it actually mean no limit since its doing an aggregation ?

johntdyer,

Not sure I follow. In the 2nd query the size:1000 seems to be set properly unless I’m missing something obvious. What does the cluster return as a response?

John…

Please refer to https://github.com/grafana/grafana/issues/15870 for further information/updates.