I have been trying to figure this out for months on alerting. I have even discussed doing enterprise licensing for Grafana but I need a problem figured out. I have a multitude of servers, services and other miscellaneous monitoring going on and need dynamic alerting. If I have 100 servers and they all have different services and logs being monitored, I should be able to dynamically get the hostname, service or whatever other issue we have that’s going into a database. The alerts know that there’s a variable there for a column:
But that’s what exactly shows up in the email:
These columns show up in Explore:
But not for alerting. I also can’t do a SELECT in the alert for the text based columns so I can’t get the info that way either.
I find it hard to believe that the mark was missed on this. Am I missing something? (calling it by label, tag, field, column, whatever) or is it true that alerting is not dynamic?
Hi!
Yes this is supported 
There is no variable called $column, but can use $labels in custom annotations to reference some or all of the labels returned by the datasource query. You can find the docs on this here.
If you are using PostgreSQL you can do something like this:
SELECT 1, id::text, domain, chassis, model, oper_state_code FROM servers WHERE oper_state_code = 'degraded';
This is how it looks:
1 Like
Okay this is my query on InfluxDB:
select domain, chassis, id, model, last(oper_state_code) from Servers where oper_state = ‘degraded’
So would I put ::text infornt of each column I want recognized as a label? I tried using label before
For InfluxDB no because ::text is a PostgreSQL feature. I’m afraid I don’t use InfluxDB so I’m not 100% sure how it should look in InfluxQL, but you can take the example which is for PostgreSQL and translate it to InfluxQL and it should just work.
Looking at the InfluxQL you’ve already shared I think you want a number (I used 1) and also need to make sure that the labels are strings. You should be able to find out how to do this with the InfluxDB documentation on the InfluxDB website.
And I don’t see anything about strings for tag keys:
tagKey
chassis
domain
host
id
location
service_profile
type
This seems to be the problem. There is no examples for $labels being used with different datasources. After talking to Grafana they said it’s not possible. So I’m not sure what to do. I am a BIG proponent of Grafana and would like to use it and replace Nagios. But with alerting not being dynamic or not properly documented/supported we can’t use it
This is the query and the data returned. We are not getting the other key tags in the query:
In MySQL the query would be SELECT 1, cast(id as char(11)) AS id, domain, chassis, model, oper_state_code FROM servers WHERE oper_state_code = 'degraded';
Looking at the InfluxDB docs tags are strings, so no need to make id a string if using InfluxDB.
What we’re seeing when we run the InfluxDB query, not all the columns are returned in the Alerts query. Could that have something to do as why we can’t get labels?
