How to set/enforce TTL for BoltDB files on Index Gateway disk?

Grafana Loki
1

Hi, I have a Loki setup running on GKE (official helm chart + microservices mode).

Right now, there are about 5 index gateway pods (in simple mode) running with each having a 20GB disk. Of late, i am receiving disk full alerts for the index gateway. The documentation doesnt have any details about TTL for the downloaded index files. The storage_config section is configured as shown below

storage_config:
  boltdb_shipper:
  active_index_directory: /var/loki/boltdb-shipper-active
  cache_location: /var/loki/boltdb-shipper-cache
  cache_ttl: 48h
  index_gateway_client:
    server_address: dns:///loki-index-gateway:9095
  query_ready_num_days: 2
  shared_store: gcs

Is the above config responsible for setting TTL on index gateway? If so, when I checked the files inside the volume, i was able to find files which were older than 1 month even (in that case, maybe the TTL is not getting enforced?). Would like some insights/help around this. Thanks!

2

Also, it would be really helpful if someone can give me any insights around disksize calculations / estimates for index gateway pods (right now, we are just increasing as the disk gets full but I was wondering if there’s a better way to do it)