"Template variables are not supported in alert queries" while setting up Alert

I am still waiting for support for just constants!

1 Like

In my company we launch several environments every day, each with its own dashboard, I can’t even express how wonderful it would be to be able to use variables in alarms (what we need most), thresholds, and gauges.

With constant variables it would be enough to automate almost the entire process.

1 Like

There should be more information about this. I also new user and try to learn grafana. Shuld not be so hard to create a alert.

1 Like

Got this problem too, I think it will be good if alerting system check “Include All option” and if it set - use “custom all value”:
image

If “Include All option” not set - then show error.

Or even you can introduce default value in template.

Supporting for constant is important and required. Grafana needs this function indeed.

1 Like

Hi there,

I have a repeating dashboard set up, based on tags (variable). I want to set up alerts for each repeated panel (which is monitoring a different service based on the tags). However, I am reading that we can’t use the variable in the alert rules.

I have hundreds of services (each service has a unique tag,) and to manually create all those panels just so each can have an alert rule set up seems like a major usability issue. If you can use “tags” as a variable for repeating panels, you should be able to use the same variables for the alerts, right? Or is there some other way I should approach this?

To clarify further, I have:

  1. A dashboard setup with multiple variables (query, tag,) which can be used to select All, multiple, or a single “service” (a tag).
  2. On this dashboard, there is a single repeating horizontal row (one row per tag). This iterates through all the service tags, and creates a row for each.
  3. I want to have an alert for each repeated graph in each repeated row, but when I use the service tag variable in the alert query it returns an empty set / null.

How else can I do this? The “repeat” feature is really useful, but it looks like the variables don’t carry over to alerting.

2 Likes

i need the same requires

1 Like

While setting up alert getting this error

1 Like

This is currently not possible. There is already a feature request for this in github.

How did you achieve this?
I have the same requirement. I have a table of devices in rows with links. on click of a device I have another dashboard where the monitoring is specific to the device selected. Now I want to set alert on the child dashboard

Would be curious as to the priority being placed on this considering that every competitor to Grafana allows for this capability. My main fear is that it will make it’s way into their paid offering before it makes its way into the Community version.

Has there been any more discussion or progress surrounding this? I know that you suggested it may not be what people want but by the looks of it, it’s what people need and it shouldn’t be too difficult to implement as long as there are some guardrails.

1 Like

We have around 500+ servers with some more than 1TB RAM.
We have prometheus node exporter running on those servers.
We need email alerts on

  1. cpu usage high
  2. ram usage high
  3. network usage high
  4. disk usage high
  5. root fs more than certain % (80)
  6. sys load > x %
  7. system down
    Multiply this 7 * 500 servers. How is this even feasible to sit there and create a separate dashboard for 500+ servers? How is this efficient to create 3,500 alerts for by going to each server’s dashboard and editing panel and creating alert?
5 Likes

You can export it as a JSON, parse it with a python script or whatever language you want, and just generate the same dashboard 500 times but with different server names. Alerting rules and every setting will remain the same. Even the requests for importing can be done in the same script.

1 Like

Exactly. How in the world is this not a priority. With the scalability of Prometheus you would think this would be high on the list, if not already implemented. Template variables in dashboards are what allows us to graph data like this in the first place. I’m not making 100 duplicate dashboards just so I can have alerts.

1 Like

Hi guys. Any news on this topic?

So any updates on this very crucial thing?

1 Like

Hi,

Any workaround ?

I’d like to get an answer on this as well. I see plenty of people complaining about this issue for years and yet nothing has been done. I’m not concerned about “why” template variables are not usable in alerts, I just want them to work.

Meaning no disrespect, but if Grafana Labs wants to crow about how great template variables are(and they are), then why in the world are they not supported in alerts?

At present I have close to 20+ alerts that fire off based on a variety of metrics. I can easily create queries with template variables but when when it comes to alert queries I can’t use them at all! Template variables are not a “nice to have” in Grafana, they are integral to setting up dashboards that are concise and consistent. It’s what makes Grafana Grafana! But you can’t use them with alerts? Asking us to do “workarounds” is not the answer. Alerts MUST be able to support template variables just like you can anywhere else in Grafana. I can’t even use a simple constant variable for the alert message as they are not interpolated in the alerts.

As Grafana Labs expands its offerings to the cloud (either via their own SaS offering or through Amazon’s Amazon Managed Grafana Service(AMG)) the most important aspect of usability, using variables in alerts is receiving little to no serious attention. I don’t know if Grafana Labs realizes what an impediment this is to its customers.

This is a huge missed opportunity for Grafana Labs so much so, that I’m looking for alternatives to Grafana (thought I’d never say that).

Should we open up a bug for this issue or make a feature request?

4 Likes

I’ve added a feature request to allow using template variables in alert queries:

2 Likes