S3 configuration with Loki

Grafana Loki
1

Good afternoon.

I have Loki version 3.2.1 configured with my Amazon S3. It saves the data in the root of my project, but I want it to save it inside a specific folder, for example, /logs/, but it’s not working.

This is the Loki code:

auth_enabled: false
server:
  http_listen_port: 3100
  log_level: warn
  grpc_listen_port: 9095
common:
  path_prefix: /loki
  storage:
    s3:
      bucketnames: ${S3_BUCKET_NAME}
      region: ${AWS_REGION}
      access_key_id: ${AWS_ACCESS_KEY_ID}
      secret_access_key: ${AWS_SECRET_ACCESS_KEY}
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory
schema_config:
  configs:
    - from: 2024-01-01
      store: tsdb
      object_store: s3
      schema: v13
      index:
        prefix: index_
        period: 24h
storage_config:
  aws:
    s3: s3://${S3_BUCKET_NAME}
    s3forcepathstyle: true
  boltdb_shipper:
    active_index_directory: /loki/index
    cache_location: /loki/boltdb-cache
limits_config:
  allow_structured_metadata: false
  reject_old_samples: true
  reject_old_samples_max_age: 168h
  max_cache_freshness_per_query: 10m
  max_query_parallelism: 32
  max_entries_limit_per_query: 1000001
  ingestion_rate_mb: 15
ingester:
  lifecycler:
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
  chunk_idle_period: 1m
  chunk_target_size: 1572864
  chunk_retain_period: 1m
compactor:
  working_directory: /loki/compactor
query_range:
  align_queries_with_step: true
  max_retries: 5
  parallelise_shardable_queries: true
table_manager:
  retention_deletes_enabled: true
  retention_period: 168h
query_scheduler:
  max_outstanding_requests_per_tenant: 1000```
2

I would recommend you to not do that. S3 bucket itself doesn’t cost anything, there should be no reason to not have a dedicated S3 bucket for Loki.

3

Thank you for the recommendation.

Even so, I’m going to prefer using folders because I will install Loki in 3 different environments (testing, developing, and production), and I prefer to keep it organized in folders, for example:
/testing/logs

So if you know how, I would appreciate your help.

4

Then you should have all of them use different S3 buckets. It’s also easier in terms of deployment.

I don’t know how to do it, and I don’t know if it’s possible. But even if it is you should still not do it, and that is my recommendation.