Grafana 8 alerting - multiple series values

[luisgarijog]

What Grafana version and what operating system are you using: grafana 8.0.3, grafana running in docker container on centos host

What are you trying to achieve?: I’m trying to use the multi dimensional rule, in a query I bring several series: several pressure sensors identified by their alias and their pressure values at different time instants. I want to be able to group them by alias and value to check if the last sample exceeds a certain value.

How are you trying to achieve it?: In the query, against a postgresql instance, I use a window function to keep the last value in time for each alias. Then I use a mathematical expression to define that if the value exceeds 10 the alert is generated.

What happened?: When previewing my alert I get an error, it tells me that: the frame cannot be uniquely identified by its tags: it has duplicate results with {} tags.

What did you expect to happen?: Since in the mathematical function of query A < 10 , ($A < 10) it does identify each series individually and returns a 1 or 0 if the value condition is met or not, I would expect this series distinction to be reflected in different labels so that the alerts are identified by this label.

Can you copy/paste the configuration(s) that you are having problems with?:

Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were. Info: invalid format of evaluation results for the alert definition : frame cannot uniquely be identified by its labels: has duplicate results with labels {}

Did you follow any online instructions? If so, what is the URL?: Create Grafana-managed alert rules | Grafana documentation

[mattabrams]

Hi @luisgarijog

This is interesting. Thanks for posting. I can replicate:

I’m wondering if this has to do with the math expression feature, which is still in beta as well and known to break.

Do you think you can accomplish your goal using the classic condition and last()? In this case the test then works and returns:

[ metric='up{instance="demo.robustperception.io:9090", job="prometheus"}' labels={__name__=up, instance=demo.robustperception.io:9090, job=prometheus} value=1 ], [ metric='up{instance="demo.robustperception.io:9091", job="pushgateway"}' labels={__name__=up, instance=demo.robustperception.io:9091, job=pushgateway} value=1 ], [ metric='up{instance="demo.robustperception.io:9093", job="alertmanager"}' labels={__name__=up, instance=demo.robustperception.io:9093, job=alertmanager} value=1 ], [ metric='up{instance="demo.robustperception.io:9100", job="node"}' labels={__name__=up, instance=demo.robustperception.io:9100, job=node} value=1 ]

[michaelosthege]

For me the classic condition was not a viable workaround, because I wanted to use template variables in the message text.

I got it to work by adding another reduce expression inbetween. See ngalert don't work with prometheus datasource · Issue #35540 · grafana/grafana · GitHub

[Skywalker]

Experiencing the same issue with postgres, Does not work to add a reduce expression in between.
Did anyone find a fix for this?