Display alert annotations on panel when alert is triggered

[hphillips2]

Hi,

I’m new to Grafana and struggling with showing alerts on my panels. I want to display the alert annotation on graph within the panel when the alert is in the “alerting” state. At the moment, annotations seem to appear when the panel (or maybe the alert?) is refreshed, not only when the alert is triggered.

My panels are just CPU and memory utilisation from my computer read from AWS Timestream. This is how I’ve configured the alert for memory utilisation (CPU is the same, just different threshold):

And this is what the graph looks like (I’ve stopped sending data through):

I only want the “alert” annotation to display when the alert is triggered, not when it’s “ok” or “pending”. At the moment, the alert seems to only show when I refresh the alerts (I think?).

[antonio]

welcome to the community @hphillips2

I’m not sure what is the logic behind alerting-triggered annotations, but you could check this API:

/api/annotations?alertId=XX

It will tell you what was the alert status of each annotation. There are different filters you can use such as alertID , panelId, etc…

Part of the API response reflecting last status change:

        "newState": "Alerting",
        "prevState": "Pending",

To filter by alertId :
If you don’t know the alertId, you can retrieve it via this other endpoint:

GET /api/v1/provisioning/alert-rules/{UID}

Note: the UID of the alert shows in the browser’s URL when viewing/editing the alert in the UI