A simple use-case for this is if we wanted to tag alert annotations. Suppose an alert fired for
panel 1 because we had an unplanned outage, and an alert fired for
panel 2 because of a scheduled maintenance.
If we have an annotation query for tags
unplanned, it makes sense that we’d only expect to see annotations with the tag
unplanned only on the panels they were created for; i.e., I don’t really care about the unplanned annotation on panel 2 because it didn’t affect panel 2’s metrics (i.e.,
panel 2 didn’t alert on anything).
To give a more robust example, we’ve added a pipeline that transforms alert annotations into regions, and tags those regions with environment variables (
prod, etc.). Our dashboard has annotation queries for these – however, since these were generated from alerts, we want that annotation to stick for the panel that alerted. As it works currently, as soon as we try to query say “all production alerts”, simply because they’re tagged as
prod, we see the annotations on all panels, not just the panels that alerted (making it quite confusing on what alerted).
The end goal is that we want to be able to tag annotations while preserving which panel that annotation alerted for.