I’m using the Bigquery datasource plugin to create an alert rule. I’m monitoring several machines that can have errors. If an error last longer than x minutes, then the alert should be sent to the contact point. The query looks like this:
SELECT
timestamp_diff(current_timestamp(), timestampOccurred, MILLISECOND) as duration,
FROM
MyTable
WHERE
status = 'open'
Now, I would like to add context information to the summary annotation, like which machine has an error, a short description of the error, etc. I have all the information in my Bigquery table.
So my question is: How can I set up the alert and include this context information to the alert message?
I’m not using bigquery, but I guess the rule should be the same.
I think that passing additional info based on the query is only achievable with multidimensional alerts (the ones that create multiple rules). You need to ensure that the data you want regarding the machine is in the series labels (see screen below):
both of my series have label server. Now, if I want to pass that label into e.g. summary, I can use templating (if you know GO, those are based on go templates), like this: Machine {{ .Labels.server }} is down! in summary / description field.
thank you for your explanation. I finally solved my issue.
The problem was, that when I queried more than one column, I got the following error message:
I interpreted it as that I cannot query more than one column at the same time and so I had no idea how to get the context information into the result. Now, I found out that alerts can’t handle timestamps. After transforming the timestamp into a string, I can access the additional columns as labels in the way you mentioned.
So, for documentation purposes, my final query looks similar to this:
SELECT
TIMESTAMP_DIFF(CURRENT_TIMESTAMP(), MAX(timestampOccurred), MINUTE) AS duration,
STRING(timestampOccurred) AS timestampOccurred,
machineName,
errorCode,
errorMessage
FROM
myTable
WHERE
status = 'open'
