Multiple thresholds in a single alert rule (Unified Alerting)

Grafana Configuration
1

Hi all,

In order to monitor a database’s tablespaces I have time series like the following:
tablespaces{tablespace_name=“SYS”}
…
tablespaces{tablespace_name=“SYSTEM”}

To build alerts,
1- first I have a simple query named insMetric:
tablespaces{}

2- then a reduce expression on insMetric called insMin:
Function: min

3- and finally a math expression, on insMetric, called insAlert:
$insMin > 20

This works fine but there’s just one threshold, 20, for all series. I would like to have multiple thresholds in a single alert rule, let’s say 20 for SYS and 25 for SYSTEM. Is this possible? How?

Thanks,
Sebastian-

2

Hi! Any ideas on this issue?

3

I would like to know it too.

4

no way unfortunately. you need to clone your alert for every threshold.

5

I handle this in the collector’s script logic. After collecting the data, the script will match names and set thresholds individually to them and output as ‘Tag fields’ for influxDB (v1.8). It must be a Tag
In Grafana’s Alertmanager I do my query and add the threshold field in “group by”, so I can access it as a Label to be used in a Template Logic

Something Like This:
SELECT alert_level
FROM oracle_alerts
WHERE alert_name =~ /Tablespaces_Used/
AND metric_value > warn_limit (or crit_limit)
AND $timeFilter
GROUP BY instance,fqhn,sub_check,warn_limit,crit_limit
^^^^^^^^^^^^^^^^^^
Thresholds are these

Then you access them using {{ labels.warn_limit }} or {{ labels.crit_limit }}, enabling you to set your severity