Currently I have a panel that holds 3 queries, each representing the system load, query as follows:
max(node_load1{job=“node-exporter”, instance="$instance"})
Where $instance is a predefined template variable that represents whatever instance the graph currently represents (we have around 15 different instances: 10.0…0.1, 10.0…0.2, etc)
This method works perfectly for graphing - as we change the instance we want to look at, the graph updates as it should. However this is using a template variable, so I can’t alert based on the query. When I try to set an alert with a query that has $instance in it, the system responds saying ‘Template variables are not supported in alert queries’.
When I changed the “$instance” variable in the query to a specific instance like “10.0…0.1” then it works and I can create an alert as I wanted to, but now (obviously) the query only responds to that one specific instance, and the graph is stuck on that instance regardless of what instance I change the panel to represent.
My question is, is there a way to use a non-template variable (that is, not $instance) but still have it be a variable that can change based on the instance that I subscribe to in the dropdown menu, so that the graph changes, and an alert system can be made?
Any help is appreciated. Thanks!