What Grafana version and what operating system are you using?
Grafana 13.0.1, running in Docker on CentOS Linux.
What are you trying to achieve?
I have 65+ million rows in the annotation table (PostgreSQL backend). I want to automatically clean up old annotations using built-in retention settings to prevent DB bloat and speed up future upgrades.
How are you trying to achieve it?
I configured the following in grafana.ini:
[annotations]
cleanupjob_batchsize = 200000
[annotations.dashboard]
max_age = 60d
max_annotations_to_keep = 1000000
[annotations.api]
max_age = 60d
max_annotations_to_keep = 1000000
I also tried adding:
[unified_alerting.state_history]
enabled = true
backend = annotations
[unified_alerting.state_history.annotations]
max_age = 60d
max_annotations_to_keep = 1000000
And restarted Grafana.
What happened?
-
No errors in logs.
-
Only this line appears:
logger=annotation.app level=info msg="Starting annotation cleanup loop" interval=24h0m0s retention=2160h0m0s -
The row count in the
annotationtable does not decrease even after 24+ hours. -
No other logs related to legacy annotation cleanup (no
level=error, no success messages).
What did you expect to happen?
Grafana should automatically delete annotations older than 60 days or keep only the newest 1M per type, especially since most of our annotations come from Unified Alerting (alert_id IS NOT NULL).
Can you copy/paste the configuration(s) that you are having problems with?
See above — full [annotations] and [unified_alerting] sections.
Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
No errors. Only the annotation.app log line (which seems unrelated to the main annotation table).