I’m new with loki and trying to understand how to correctly set retention for it. I currently have loki in docker setup with such config:
auth_enabled: false server: http_listen_port: 3100 common: path_prefix: /loki storage: filesystem: chunks_directory: /loki/chunks rules_directory: /loki/rules replication_factor: 1 ring: kvstore: store: inmemory schema_config: configs: - from: 2020-10-24 store: boltdb-shipper object_store: filesystem schema: v11 index: prefix: index_ period: 24h table_manager: retention_deletes_enabled: true retention_period: 24h ruler: alertmanager_url: http://localhost:9093
After a month of working on host, loki consumed nearly 300 Gb of space and created ~200 000 chunks. So I wanted to reduces this number of chunks and used space somehow. I tried to add Compactor with such config.
compactor: working_directory: /loki/retention shared_store: filesystem compaction_interval: 10m retention_enabled: true retention_delete_delay: 2h retention_delete_worker_count: 150
And I see in log that it already run 3 times, but it always compacts the same tables - starting with 19420 and ending with 19450
level=info ts=2023-04-04T10:42:30.779132767Z caller=compactor.go:495 msg="finished compacting table" table-name=index_19420 .... level=info ts=2023-04-04T10:42:30.784918253Z caller=compactor.go:495 msg="finished compacting table" table-name=index_19450
So, not really understand why it’s not looping over all data and tries to optimize it?
I added also limits_config: retention_period: 744h to main config but gave no effect.
Will be glad to see some basic explanation for a newbie like me.
Thanks in advance!