Dynamic labels for alerting for multiple servers

Hi

Grafana - 10.4.11
Datasource - Prometheous - 2.33.3
OS - linux - 22.04

Suppose we have to monitor CPU and memory alert for 100 servers so is there any way to make alerts with some variable or annotation or labels

Is it possible?

If possible then can anyone please provide the link so it would be better for us

Thanks in advance

Hey @ekta291, ideally the alert rule query should return a label that identifies each server instance.

In this example, the query returns the instance and probe labels. They are referred to as data-source query labels in this doc.

This label will also be used to identify your alert instance. Read here about creating multiple alert instances from your alert rule.

This example uses the cpu label to distinguish the alert instances.

You can then use that label for routing your alerts, or use the label in the alert rule annotation and notification using templates.

Hope it helps!