Hi, I have a Loki setup running on GKE (official helm chart + microservices mode).
Right now, there are about 5 index gateway pods (in simple mode) running with each having a 20GB disk. Of late, i am receiving disk full alerts for the index gateway. The documentation doesnt have any details about TTL for the downloaded index files. The
storage_config section is configured as shown below
storage_config: boltdb_shipper: active_index_directory: /var/loki/boltdb-shipper-active cache_location: /var/loki/boltdb-shipper-cache cache_ttl: 48h index_gateway_client: server_address: dns:///loki-index-gateway:9095 query_ready_num_days: 2 shared_store: gcs
Is the above config responsible for setting TTL on index gateway? If so, when I checked the files inside the volume, i was able to find files which were older than 1 month even (in that case, maybe the TTL is not getting enforced?). Would like some insights/help around this. Thanks!