I am using Grafana v11.5 and have built dashboards for a while. I am familiar with dashboard panels. I am venturing into new horizons to take advantage of Alerting and MQTT integration in an attempt to build an orchestrated workflow based on an alarm condition. Here is what I am trying to accomplish:
I have an enterprise sensor network made up of remote sensors. Each sensor produces a status json payload sent to the broker. I organize my sensors in their own topic areas like:
sensors/<sensor_id>/status
sensors/<sensor_id>/<measurement_element>
I have a panel with an InfluxDB query with indicators in the dashboard when a sensor fails to report and for how long. I built an alert with a threshold defined and working. I am trying to integrate to the contact AlertManager MQTT handler and post to the topic (conceptually):
sensors/offline/<sensor_id>
using the alert query and label for the sensor. Is this possible? I would use this notice to kick off an orchestration response involving other enterprise assets.
I have read about using labels in notification payloads but have not seen anything written about labels used for, say, email subject lines or MQTT topics in the contacts integration setup. Any insights for how to use the alert return label for this would be appreciated.
Hi @cschmidt5,
The broker URL and Topic fields in the MQTT integration are fixed values. Currently, only the Message field supports notification templates.
If you can use a general topic for all sensors, you can define sensor_id as a custom label (rather than an annotation, as explained below). This label will then be passed to the notification message.
Keep in mind:
- alert rule templates differs a bit from notification templates, which are used in the MQTT Message field.
- notifications can include one or multiple alerts
- multiple alerts may have different
sensor_id, which should be avoided in your case. A notification should only group alerts from the same sensor_id.
Suggested approach:
-
Create a sensor_id label and define a label template to set the sensor_id value from the query
-
Configure a notification policy that groups all alerts by sensor_id busing the Group by option.
-
Create a custom notification template for the Message option and use {{.CommonLabels.sensor_id}} to include the sensor_id value.
I’m not sure if this fully aligns with your use case, but hopefully, it helps or points you in the right direction!
Thank you for your detailed response. After multiple attempts in trial and error, I eventually concluded the same as you describe. I ended up creating fixed topic areas for each of my sensors and have the alert side working with custom notifications. For my use case, it does fall a bit short for the automation and orchestration I was looking to do; however, it does provide me the threshold deviation I need on the front end. Grafana has made lots of improvements and has incorporated many new features since I was first exposed to it. Perhaps a future version will include improvements to the framework and its various integrations to expose more dynamic variables. Anyway, always a pleasure to get such informative feedback through community forums.
