In grafana, we had created a variable to list applications for filtering,
while we filter any one application to get unique count, it by default includes other 2 apps count and display value as “3”. This may be due to cardinality aggregation feature in ES, But we wanted to filter and display value as “1”. is it feasible?
Yeah you can do this. Like you have created variable with the name of application. Put this variable in your query in where clause like
where application in $Variablename ( Name which you have assign to the variable). I wish this will will be helpful.
Thanks, but our team expect to see response value as “1”. Is there any alternate options that I can use to get this exact unique count (ignoring combination) as “1” at elasticsearch level or in Grafana?
I will check on using Hash (also need to check its flexibility with automated deployment scenarios)
From two ES forum references which you shared, I could try 1st reference and check if below option may be helpful. But it may be challenging to handle via grafana template variable filter. Also we end up ensuring that our filter or search always match expected response structure.
Thanks @b0b. Initially I was planning to use this process metadata. But refrained and used fields for unique tracking of certain additional details like application etc. What would be the major difference between metadata & fields? Could you please clarify? or please share reference to understand the difference in better way.
Not sure how well this works for metricbeat… Metadata is mostly internal only to the service in question.
When I use metadata fields in Logstash I can set them on different inputs for different kinds of logs. Then I use the metadata to set to which Elasticsearch index the messages will be routed. My Logstash output looks like this
application - for Hash
application1 - array retained to handle “template variable” filtering only
But it is not helpful while I try to get count dynamically with filter option (from array based field
only for template variable usage) as referred below
I should have written “Group by” instead of grouping
The third row in the query editor. With a short interval when I used “Group by” Date Histogram I got a float instead of 0 or 1. I guess it was the average over the time range when split into interval sized buckets, if that makes sense…
This is unfortunately not a problem I personally need solving at the moment and I don’t have time for more testing as I have other proprieties…
As I mentioned before, I have no direct experience of doing exactly what you are so everything I have written have been theoretical suggestions of what could work.
Good luck Hopefully you get it to work the way you expect it to.