Loki disk space inflation

I don’t understand the space disk inflation from loki.

I have the settings like this

  max_look_back_period: 96h

  retention_deletes_enabled: true
  retention_period: 96h

Anyway if I check in the container I can see in the folder /tmp/loki/chunks
In december with a 96h retention period, I have files from May, June, July…

Why, is there any clear and clever answer on this topic somewhere or this is a product functionnality: you keep the files from the beginning but you can request just the retention period?

If you are using recent version of Loki you’ll want to use compactor instead of table manager. See Retention | Grafana Loki documentation.

Yeah, I’ve spent the whole day on it yesterday.

The process is a good joke.

So you have to use compactor… But compactor does not delete the files, compactor is just here for compaction of index files and applying log retention…

The chunks files are not deleted while applying the retention algorithm on the index. They are deleted asynchronously by a sweeper process, yes, NOT the compactor !

Despite this previous indication, we also wan read:
retention_delete_delay is the delay after which the Compactor will delete marked chunks.”

Does the compactor delete the marked chunks ?

Who kows, cause the best part of the joke is here : “They are deleted asynchronously by a sweeper process and this delay can be configured by setting -compactor.retention-delete-delay .”

So the compactor deletes the marked chunks, you have a parameter for this, but in reality the compactor does not delete the chunks (the sweeper does).

Yes, it means you need a parameter for the compactor that is not in charge of deletion, to activate the secret sweeper process in charge of deletion.

I am very happy with this fucking thing, very happy with the documentation also. I’ve understood documentation is not important, it’s better to spend time developing new features instead of wasting the time in sub tasks like writing documentation.

It is also important to define parameters for a task, used by another task: this is the way to simplify the whole process. Yes, this is the way.

So what, how can I have a file deletion?

auth_enabled: false

  http_listen_port: 3100
  grpc_listen_port: 9096

  path_prefix: /tmp/loki
      chunks_directory: /tmp/loki/chunks
      rules_directory: /tmp/loki/rules
  replication_factor: 1
      store: inmemory

  - from: 2023-02-08
    store: boltdb-shipper
    object_store: filesystem
    schema: v11
      prefix: index_
      period: 24h

  # working_directory is the directory where marked chunks and temporary tables will be saved.
  working_directory: /tmp/loki/compactor
  compaction_interval: 10m
  retention_enabled: true
  # retention_delete_delay is the delay after which the Compactor will delete marked chunks.
  retention_delete_delay: 10m
  # retention_delete_worker_count specifies the maximum quantity of goroutine workers instantiated to delete chunks
  retention_delete_worker_count: 150

  alertmanager_url: http://localhost:9093 

  max_concurrent: 2048

  max_outstanding_per_tenant: 2048
  compress_responses: true

  max_look_back_period: 12h