Answering my own question from ~7 months ago…
Assume we have five fluid tanks and that each tank has a temperature controller with a setpoint value that is stored in InfluxDB and assume that each tank has a different setpoint, where we always need to be within 3 degrees of the setpoint.
We want to create one multidimensional alert that will cover each unique scenario for each tank, triggering an alert whenever any tank’s temperature moves beyond its unique allowable range.
To better visualize this, here is a table representing our five tanks, their temperature setpoints, and their allowable range:
Tank |
Setpoint |
Allowable Range (±3) |
A5 |
45 |
42 to 48 |
B4 |
55 |
52 to 58 |
C3 |
60 |
57 to 63 |
D2 |
72 |
69 to 75 |
E1 |
80 |
77 to 83 |
We can create a single multidimensional rule to cover all 5 tanks, and we can use Flux to compare the setpoint and actual value for each tank. In other words, one multidimensional alert can monitor 5 separate tanks, each with different setpoints and actual values, but all with one common “allowable threshold” (i.e. a temperature difference of ±3 degrees).
In our data query, we must add extra functions to get our data into the proper format, including a pivot()
, map()
, rename()
, keep()
, and drop()
function:
from(bucket: "HyperEncabulator")
|> range(start: v.timeRangeStart, stop: v.timeRangeStop)
|> filter(fn: (r) => r["_measurement"] == "TemperatureData")
|> filter(fn: (r) => r["MeasType"] == "actual" or r["MeasType"] == "setpoint")
|> filter(fn: (r) => r["Tank"] == "A5" or r["Tank"] == "B4" or r["Tank"] == "C3" or r["Tank"] == "D2" or r["Tank"] == "E1")
|> filter(fn: (r) => r["_field"] == "Temperature")
|> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)
|> pivot(rowKey:["_time"], columnKey: ["MeasType"], valueColumn: "_value")
|> map(fn: (r) => ({ r with _value: (r.setpoint - r.actual)}))
|> rename(columns: {_value: "difference"})
|> keep(columns: ["_time", "difference", "Tank"])
|> drop(columns: ["actual", "setpoint"])
|> yield(name: "mean")
Note in the above that we are calculating the difference between the actual and the setpoint. The way Grafana parses the result from InfluxDB is that if a _value
column is found, it is assumed to be a time-series. The quick workaround is to add the following rename()
function:
|> rename(columns: {_value: "something"})
The above query results in this time series:
Next, add an expression to the Grafana Alert rule by creating a Reduce expression for the above query to reduce each of the above to a single value. This value represents the temperature differential between each tank’s setpoint and its actual real-time temperature:
Next create a math expression to be alerted on by creating a condition that checks if the absolute value of our reduce calculation is greater than 3, abs($(B))>3.0
:
We can now see that two tanks, D2
and E1
, are evaluating to true. When we preview the alert we can see that those two tanks will trigger a notification and change their state from Normal
to Alerting
: