I use Grafana with a separate Prometheus datasource for each datacenter. I’ve found template variables very useful, to be able to write a single dashboard that can be used for each DC. Similarly, I’d like to write a set of alerts, and have them apply to each datacenter. I found https://github.com/grafana/grafana/issues/6557, which indicates that alerting + template variables are a thorny problem, and may not be solved soon.
In the meantime, what’s the best workaround? I’m thinking of creating a single dashboard for alerts, set to a specific datacenter, then exporting it to JSON, committing it to our configuration repo, and turning it into a separate dashboard for each datacenter using sed. Is there a better way?