Loki retention isn't working as expected—logs older than 24 hours are not getting deleted

Grafana Loki
1

Hi, I have below configuration for grafana loki . I am expecting that the chunks will be deleted after every 24hrs. But it is not working as expected.

auth_enabled: false

server:
http_listen_port: 3100
http_server_read_timeout: 600s
http_server_write_timeout: 600s
grpc_server_max_recv_msg_size: 18388608
grpc_server_max_send_msg_size: 18388608

common:
path_prefix: /loki
storage:
filesystem:
chunks_directory: /loki/chunks
rules_directory: /loki/rules
replication_factor: 1
ring:
kvstore:
store: inmemory

schema_config:
configs:
- from: 2020-10-24
store: tsdb
object_store: filesystem
schema: v11
index:
prefix: index_
period: 24h

storage_config:
tsdb_shipper:
active_index_directory: /loki/index
cache_location: /loki/index_cache
cache_ttl: 24h

limits_config:
enforce_metric_name: false
reject_old_samples: true
reject_old_samples_max_age: 24h
ingestion_rate_mb: 50
ingestion_burst_size_mb: 100
max_global_streams_per_user: 5000
per_stream_rate_limit: 50MB
retention_period: 24h

compactor:
retention_enabled: true
shared_store: filesystem
working_directory: /loki/compactor-work

chunk_store_config:
max_look_back_period: 0s

table_manager:
retention_deletes_enabled: true
retention_period: 24h

ingester_client:
remote_timeout: 15s

2

Try disabling table manager (see Log retention | Grafana Loki documentation).

3

Thanks for the response, tried that but still not working

This is my updated configuration:

auth_enabled: false

server:
http_listen_port: 3100
http_server_read_timeout: 600s
http_server_write_timeout: 600s
grpc_server_max_recv_msg_size: 18388608
grpc_server_max_send_msg_size: 18388608

common:
path_prefix: /loki
storage:
filesystem:
chunks_directory: /loki/chunks
rules_directory: /loki/rules
replication_factor: 1
ring:
kvstore:
store: inmemory

schema_config:
configs:
- from: 2020-10-24
store: tsdb
object_store: filesystem
schema: v11
index:
prefix: index_
period: 24h

storage_config:
tsdb_shipper:
active_index_directory: /loki/index
cache_location: /loki/index_cache
cache_ttl: 24h

limits_config:
enforce_metric_name: false
reject_old_samples: true
reject_old_samples_max_age: 24h
ingestion_rate_mb: 50
ingestion_burst_size_mb: 100
max_global_streams_per_user: 5000
per_stream_rate_limit: 50MB

compactor:
working_directory: /loki/compactor-work
compaction_interval: 10m
retention_enabled: true
retention_delete_delay: 1h
shared_store: filesystem
retention_delete_worker_count: 5

ingester_client:
remote_timeout: 15s

4
  1. You are missing retention_period on your latest config.
  2. You may want to set retention_period to 48h. The minimum index period for Loki is 24h, and there is a chance compactor won’t delete anything until 48h is up. I don’t set my retention this low, so I am not entirely sure what the behavior is here.