How to config multi S3 buckets in Loki S3 with multi bucket access key and secret

Grafana Loki
,
1

This is a sample config in the Loki doc:

loki:
  schemaConfig:
    configs:
      - from: "2024-04-01"
        store: tsdb
        object_store: s3
        schema: v13
        index:
          prefix: loki_index_
          period: 24h
  storage_config:
    aws:
      region: <AWS region your bucket is in, for example, `eu-west-2`>
      bucketnames: <Your AWS bucket for chunk, for example, `aws-loki-dev-chunk`>
      s3forcepathstyle: false
  pattern_ingester:
      enabled: true
  limits_config:
    allow_structured_metadata: true
    volume_enabled: true
    retention_period: 672h # 28 days retention
  querier:
    max_concurrent: 4

  storage:
    type: s3
    bucketNames:
        chunks: <Your AWS bucket for chunk, for example, `aws-loki-dev-chunk`>
        ruler: <Your AWS bucket for ruler, for example,  `aws-loki-dev-ruler`>
        admin: <Your AWS bucket for admin, for example,  `aws-loki-dev-admin`>
    s3:
      # s3 URL can be used to specify the endpoint, access key, secret key, and bucket name this works well for S3 compatible storages or if you are hosting Loki on-premises and want to use S3 as the storage backend. Either use the s3 URL or the individual fields below (AWS endpoint, region, secret).
      s3: s3://access_key:secret_access_key@custom_endpoint/bucket_name
      # AWS endpoint URL
      endpoint: <your-endpoint>
      # AWS region where the S3 bucket is located
      region: <your-region>
      # AWS secret access key
      secretAccessKey: <your-secret-access-key>
      # AWS access key ID
      accessKeyId: <your-access-key-id>
      # AWS signature version (e.g., v2 or v4)
      signatureVersion: <your-signature-version>
      # Forces the path style for S3 (true/false)
      s3ForcePathStyle: false
      # Allows insecure (HTTP) connections (true/false)
      insecure: false
      # HTTP configuration settings
      http_config: {}

deploymentMode: SimpleScalable

backend:
  replicas: 3
read:
  replicas: 3
write:
  replicas: 3

# Disable minio storage
minio:
  enabled: false

In my org, I have multi S3 compatible buckets, each bucket has its own access key/secret, those buckets don’t share access key/secert.

There is multi place to input the bucket in the config, but only one place to input accessKeyId/secretAccessKey.

So I don’t know how to config my buckets and its key/secret.

For example, I have

  • Bucket-chunk-accessKey1-Secret1,
    Bucket-ruler-accessKey2-Secret2,

  • Bucket-admin-accessKey3-Secret3,

I don’t know how to config the accessKey2, accessKey3…..

2

I don’t personally do this, so I could be wrong, you should be able to define 1 bucket with one storage config block. Also I don’t use the official Loki helm chart, if you are using that it’s also possible that it’s not supported there.

Something like this:

storage_config:
  bucket_1:
    s3: s3://<S3_endpoint>
    bucketnames: bucket_1
    access_key_id: <***>
    secret_access_key: <***>
  bucket_2:
    s3: s3://<S3_endpoint>
    bucketnames: bucket_2
    access_key_id: <***>
    secret_access_key: <***>

If that doesn’t work, then the only solution is to consider consolidating your access control. In my opinion it’s rather strange to have one access key for a bucket, unless you have a special use case. Usually identity should be tied to the entity attempting the access, not the object being accessed. But I digress.