Grafana Alert send not all affected series

[robert8103]

Hi I’m facing currently a problem with the slack alert.
I have got a dashboard with multiple series.
If the alert rule triggers for less or equal to 6 series everything is fine.
If there are more than 6 I only get the first 6 series in the slack channel shown.
Is there any solution for this?

[habib1]

hello please have a look here: Display “This Month” with relative time - #11 by castillo92 this could be related to your answer

[robert8103]

Hi @habib1, that is not exactly what I meant.
Here is the chart:

And the Slack Alert looks like this:

series1         series3
6.5              0.016666666666667

series4         series5
5.5             0.5

series6         series7
4               1

I would expect all 10 series in the slack alert that are below the limit not only the first 6.

[svetb]

@robert8103 I don’t believe that triggering of alerts on a per-series basis is currently supported. This comment by Torkel indicates that it wasn’t back in 2017, and the corresponding Github issue is still open (with more than a couple of interested parties).

Unless my understanding of the implementation is outdated, I’m actually surprised to hear that you’re getting alerts for multiple series at all. Normally what happens is that an alert is triggered as soon as one series crosses the threshold, and while that alert is active no others are triggered (even if other series also cross the threshold). I think the comments in the links probably explain this even better.

[robert8103]

Thanks @svetb for the links.
I went through the links and comments.
I was maybe still unprecise.
I’m fine with how the alerting is working.
I just wonder that only the first 6 series are shown in the alert message and if there is a way to change this to see all in the alert message.

[svetb]

Ah yes, sorry - your description is actually very clear, and on second reading I totally see the issue you’re referring to.

I don’t have the answer, but it may be a bug/limitation in the functionality. If googling doesn’t turn up anything, you may consider opening a GitHub issue (after checking that one hasn’t already been filed) in order to get more eyes on this.

[robert8103]

Hi, I created a bug for this.
Let’s see what comes around.

Grafana Alert does not send all affected series