Multi Dimensional Alerting with Graphite

Grafana v9.4.3

I’m trying to create a multi-dimensional alert rule using Graphite as the data source. I can get the alert to work fine for a single series, but not for multiple series.

Here are the results of the Graphite query. I’m aliasing them using aliasByNode to alias the metric by its unique id

However in the ‘B’ expression when I reduce each series, they’re just labeled as series 0, series 1, series 2 and so on. I can see it grabs the right value for each series, but I would expect the series to be labeled by whatever I’m aliasing it with

If I save this alert rule and get it to fire, I get an error saying - invalid format of evaluation results for the alert definition : frame cannot uniquely be identified by its labels: has duplicate results with labels {}

Any help would be much appreciated.

Welcome @lsamuel to the Grafana forum.

In just a regular time series graph (not the alerting section), could you please post query A (showing all the options available) and a screenshot showing the legend? My feeling is that the alias needs some adjusting.

Looking for something like this:

Thanks @grant2 for the response!

Here is query A from the panel, not the alerting section

I’m just aliasing it by the wildcard

In your alerting query A, what happens if you replace the wildcard (*)…

with this:

Does the reduce function (B) and threshold (C) work with the series’ names and is an error message still appearing?

Hi, so, that is basically what I had in query A to begin with in my initial screenshot. I just had the metric blocked out.

Here it is again. You can see they’re the same. The only difference is replacing $env with the actual value since variables are not supported in alerting.

But yes, this still produces the error and you can see the series are not labeled how I’d expect them to be. They should all be the 4-digit IDs

As an update, I tested this out with a different Graphite data source and I’m no longer encountering the issue. The data that is returned from the new data source has a tags key and so is able to properly label each series. My data source, however, does not have the tags key. Is this something that can be fixed on Grafana’s end? Or what do I need to do on my end to have this tags key? Thank you!