Hi,
I am trying to enable compaction and retention policies via compactor config explained here:
Here is the config:
loki:
schemaConfig:
configs:
- from: 2024-07-16
store: tsdb
object_store: s3
schema: v13
index:
prefix: loki_index_{{ .Values.cluster.nickname }}_
period: 24h
compactor:
working_directory: /tmp/data/compactor
compaction_interval: 120m
retention_enabled: true
retention_delete_delay: 2h
retention_delete_worker_count: 150
delete_request_store: s3
But getting the following error:
level=info ts=2024-10-14T17:44:24.001968636Z caller=main.go:126 msg="Starting Loki" version="(version=release-3.1.x-89fe788, branch=release-3.1.x, revision=89fe788d)"
level=info ts=2024-10-14T17:44:24.002005262Z caller=main.go:127 msg="Loading configuration file" filename=/etc/loki/config/config.yaml
level=info ts=2024-10-14T17:44:24.00261723Z caller=server.go:352 msg="server listening on addresses" http=[::]:3100 grpc=[::]:9095
level=info ts=2024-10-14T17:44:24.003751321Z caller=memberlist_client.go:435 msg="Using memberlist cluster label and node name" cluster_label= node=wek8s-grafana-loki-distributed-compactor-1-a906af5b
level=info ts=2024-10-14T17:44:24.004918098Z caller=memberlist_client.go:541 msg="memberlist fast-join starting" nodes_found=1 to_join=4
level=info ts=2024-10-14T17:44:24.044090829Z caller=memberlist_client.go:561 msg="memberlist fast-join finished" joined_nodes=13 elapsed_time=39.168688ms
level=info ts=2024-10-14T17:44:24.044124426Z caller=memberlist_client.go:573 phase=startup msg="joining memberlist cluster" join_members=loki-memberlist
level=info ts=2024-10-14T17:44:24.071687119Z caller=memberlist_client.go:580 phase=startup msg="joining memberlist cluster succeeded" reached_nodes=13 elapsed_time=27.556272ms
level=info ts=2024-10-14T17:44:26.633805198Z caller=module_service.go:82 msg=starting module=analytics
level=info ts=2024-10-14T17:44:26.633879104Z caller=module_service.go:82 msg=starting module=runtime-config
level=info ts=2024-10-14T17:44:26.633832016Z caller=module_service.go:82 msg=starting module=server
level=info ts=2024-10-14T17:44:26.633973229Z caller=module_service.go:82 msg=starting module=memberlist-kv
level=info ts=2024-10-14T17:44:26.634072365Z caller=module_service.go:82 msg=starting module=compactor
level=info ts=2024-10-14T17:44:26.634180553Z caller=basic_lifecycler.go:297 msg="instance not found in the ring" instance=wek8s-grafana-loki-distributed-compactor-1 ring=compactor
level=info ts=2024-10-14T17:44:26.634193153Z caller=basic_lifecycler_delegates.go:63 msg="not loading tokens from file, tokens file path is empty"
level=info ts=2024-10-14T17:44:26.634299625Z caller=compactor.go:410 msg="waiting until compactor is JOINING in the ring"
level=info ts=2024-10-14T17:44:26.634314034Z caller=compactor.go:414 msg="compactor is JOINING in the ring"
level=info ts=2024-10-14T17:44:27.635336805Z caller=compactor.go:424 msg="waiting until compactor is ACTIVE in the ring"
level=info ts=2024-10-14T17:44:27.75315986Z caller=compactor.go:428 msg="compactor is ACTIVE in the ring"
level=info ts=2024-10-14T17:44:27.75323114Z caller=loki.go:508 msg="Loki started" startup_time=3.762827116s
level=info ts=2024-10-14T17:44:57.754219685Z caller=compactor.go:489 msg="this instance has been chosen to run the compactor, starting compactor"
level=info ts=2024-10-14T17:44:57.754295371Z caller=compactor.go:518 msg="waiting 2h0m0s for ring to stay stable and previous compactions to finish before starting compactor"
level=info ts=2024-10-14T17:45:02.753435164Z caller=compactor.go:498 msg="this instance should no longer run the compactor, stopping compactor"
level=info ts=2024-10-14T17:45:02.753490176Z caller=compactor.go:503 msg="compactor stopped"
ts=2024-10-14T17:45:02.753509097Z caller=spanlogger.go:109 level=info msg="building table names cache"
ts=2024-10-14T17:45:02.753687375Z caller=spanlogger.go:109 level=info msg="table names cache built" duration=139.953µs
level=error ts=2024-10-14T17:45:02.753696137Z caller=cached_client.go:189 msg="failed to build table names cache" err="RequestCanceled: request context canceled\ncaused by: context canceled"
ts=2024-10-14T17:45:02.753723327Z caller=spanlogger.go:109 level=info msg="building table names cache"
ts=2024-10-14T17:45:02.753774198Z caller=spanlogger.go:109 level=info msg="table names cache built" duration=46.854µs
level=error ts=2024-10-14T17:45:02.75377992Z caller=cached_client.go:189 msg="failed to build table names cache" err="RequestCanceled: request context canceled\ncaused by: context canceled"
ts=2024-10-14T17:45:02.75378782Z caller=spanlogger.go:109 level=info msg="building table names cache"
Grafana v11.2.0
Loki: 6.10.0 (helm-chart) AppVersion: 3.1.1