Query Multiple Instances

Currently I have a panel that holds 3 queries, each representing the system load, query as follows:

max(node_load1{job=“node-exporter”, instance="$instance"})

Where $instance is a predefined template variable that represents whatever instance the graph currently represents (we have around 15 different instances: 10.0…0.1, 10.0…0.2, etc)
This method works perfectly for graphing - as we change the instance we want to look at, the graph updates as it should. However this is using a template variable, so I can’t alert based on the query. When I try to set an alert with a query that has $instance in it, the system responds saying ‘Template variables are not supported in alert queries’.

When I changed the “$instance” variable in the query to a specific instance like “10.0…0.1” then it works and I can create an alert as I wanted to, but now (obviously) the query only responds to that one specific instance, and the graph is stuck on that instance regardless of what instance I change the panel to represent.

My question is, is there a way to use a non-template variable (that is, not $instance) but still have it be a variable that can change based on the instance that I subscribe to in the dropdown menu, so that the graph changes, and an alert system can be made?

Any help is appreciated. Thanks!

Hi there!

I have run across the same issue.

Also per my knowledge, at present the template variables are not supported in the alerting.

Is there a way around this?

I tried using regex for this, the query inspector doesn’t flag this but I don’t get any data for the graph.

Some googling around this coupled with hits and trials, I was able to solve this issue.

Case 1:
node0, node1, node2…node 9
max(node_load1{job=“node-exporter”, instance=“node[0-9]”})

Case 2:
node0, server1, host3 (regex is not easy here)
max(node_load1{job=“node-exporter”, instance=“node0|server1|host3”})