A Problem on the Saving Time of Loki Chunks with S3 Minio as the Storage Backend

Grafana Loki
1

My question is, if I want to set the log to only be saved for 180 days, what should I do?
This is my current configuration for Loki：

    schema_config:
      configs:
      - chunks:
          period: 24h
          prefix: loki_chunks_
        from: "2020-09-07"
        index:
          period: 24h
          prefix: loki_index_
        object_store: s3
        row_shards: 16
        schema: v13
        store: tsdb

    storage_config:
      aws:
        bucketnames: chunks,chunks2
        s3: http://Sxxxx7:3lQxxxxxtb@10.0.0.57:9000
        s3forcepathstyle: true
      index_queries_cache_config:
        memcached_client:
          addresses: dnssrv+_memcached-client._tcp.loki-distributed-memcached-index-queries.loki.svc.cluster.local
          consistent_hash: true
      tsdb_shipper:
        active_index_directory: /var/loki/index
        cache_location: /var/loki/index_cache
        cache_ttl: 24h

    table_manager:
      retention_deletes_enabled: true
      retention_period: 4032h

table_manager.retention_deletes_enabled=true and table_manager.retention_period=4032h

Can these two parameters ensure that they are only saved for 4032 hours,Data exceeding 4032 hours will be deleted, including chunks and indexes in the minio bucket?

Do we still need to configure the lifecycle strategy of the minio bucket?

Do you have any senior who can help answer this question? Thank you very much!

2

You’ll want to use compactor instead of table manager. See Log retention | Grafana Loki documentation.

3

Also, it may be a good idea to configure a lifecycle strategy for minio still, just in case.

Compactor uses persistent volume to keep a list of chunks to delete, and it is possible to lose that file if you aren’t using persistent volume for compactor and the container is cycled. For us, in S3 we have versioning enabled (just in case someone touches the S3 bucket manually or did something incorrectly), so that chunks or index files deleted are moved into versioned history, then we configure additional lifecycle policy to remove all versioned files after 10 additional days.

4

Your answer is very good! But I’m confirming with you.

Based on your suggestion, my understanding is:
If my table_manager is set to 168h, retention_period: 168h.
So my S3 minio lifecycle management can be set to be deleted after 168h+10 days.

This is a good recommendation, is my understanding correct?

5

You are correct :+1:t4:. Even in documentation have this mentioned:

If you have a lifecycle policy configured on the object store, please ensure that it is longer than the retention period.