Loki logs not appearing even if retention_preiod is set to 168h

Grafana Loki
1

Hello,
We are unbale to see older logs more than 3 hrs even if we have set the retention period to 168h. Please help us with loki-loki-distributed config file where we can set configuration and see logs before 3hrs. This is the current setting we have in place.
limits_config:
deletion_mode: filter-and-delete
enforce_metric_name: false
ingestion_burst_size_mb: 20
ingestion_rate_mb: 10
max_cache_freshness_per_query: 10m
reject_old_samples: true
reject_old_samples_max_age: 168h
retention_period: 168h
split_queries_by_interval: 15m

2
  1. Make sure your chunk files are actually being written to your block storage.
  2. Make sure your queriers have permission to read from the block storage.
  3. Check querier logs for errors.

Also please share configuration.

3

Hello,
Thank you for your reply. Sure here is the config for loki-loki-distributed.
Also we have this deployed in our AKS with loki-distributed helmchart

auth_enabled: false
chunk_store_config:
  max_look_back_period: 0s
common:
  compactor_address: http://loki-loki-distributed-compactor:3100
compactor:
  retention_enabled: true
  shared_store: azure
distributor:
  ring:
    kvstore:
      store: memberlist
frontend:
  compress_responses: true
  log_queries_longer_than: 5s
  tail_proxy_url: http://loki-loki-distributed-querier:3100
frontend_worker:
  frontend_address: loki-loki-distributed-query-frontend-headless:9095
ingester:
  chunk_block_size: 262144
  chunk_encoding: snappy
  chunk_idle_period: 30m
  chunk_retain_period: 1m
  lifecycler:
    ring:
      kvstore:
        store: memberlist
      replication_factor: 1
  max_transfer_retries: 0
  wal:
    dir: /var/loki/wal
ingester_client:
  grpc_client_config:
    grpc_compression: gzip
limits_config:
  deletion_mode: filter-and-delete
  enforce_metric_name: false
  ingestion_burst_size_mb: 20
  ingestion_rate_mb: 10
  max_cache_freshness_per_query: 10m
  reject_old_samples: true
  reject_old_samples_max_age: 168h
  retention_period: 168h
  split_queries_by_interval: 15m
memberlist:
  join_members:
  - loki-loki-distributed-memberlist
query_range:
  align_queries_with_step: true
  cache_results: true
  max_retries: 5
  results_cache:
    cache:
      embedded_cache:
        enabled: true
        ttl: 24h
ruler:
  alertmanager_url: https://alertmanager.xx
  external_url: https://alertmanager.xx
  ring:
    kvstore:
      store: memberlist
  rule_path: /tmp/loki/scratch
  storage:
    local:
      directory: /etc/loki/rules
    type: local
runtime_config:
  file: /var/loki-distributed-runtime/runtime.yaml
schema_config:
  configs:
  - from: "2020-09-07"
    index:
      period: 24h
      prefix: loki_index_
    object_store: azure
    schema: v11
    store: boltdb-shipper
server:
  http_listen_port: 3100
  grpc_server_max_recv_msg_size: 104857600
  grpc_server_max_send_msg_size: 104857600
storage_config:
  azure:
    account_key: ABCD
    account_name: storageaccountname
    container_name: loki
  boltdb_shipper:
    active_index_directory: /var/loki/index
    cache_location: /var/loki/cache
    cache_ttl: 168h
    shared_store: filesystem
table_manager:
  retention_deletes_enabled: false
  retention_period: 0s
4

image
image1269×550 37.1 KB

If you look at this screenshot, it is not even showing logs for the past 24 hours. Any help on this will be helpful