Requirement: have multiple folders (based on tenant-id) in s3 chunks bucket, while basic-auth & multi-tenancy is implemented in Loki
Output: With the otel exporter sending logs with correct X-Scope-OrgID & Grafana datasource also has the same http header, logs in s3 buckets are being stored inside the folder with the name of the basic-auth user (123) rather than tenant-ids (abc and xyz)
Desired outcome: S3 chunk bucket should have folders with name of tenant-ids (e.g. abc & xyz)
Configuration:
I have my opentelemetry, loki & grafana setup where I’m picking up stdout app logs using the filelog receiver and sending to loki using the otlphttp exporter.
My otel collector configuration is:
receivers:
filelog:
include:
- /var/log/pods/*/*/*.log
include_file_name: false
include_file_path: true
processors:
batch:
timeout: 60s
exporters:
otlphttp:
endpoint: http://loki-gateway:3100/otlp
headers:
X-Scope-OrgID: "abc"
Authorization: "Basic foobar"
service:
pipelines:
logs:
exporters:
- otlphttp
processors:
- batch
receivers:
- filelog
My loki configuration has multi-tenancy & basic auth enabled. Loki config is:
loki:
tenants:
- name: abc
- name: xyz
schemaConfig:
configs:
- from: "2024-04-01"
store: tsdb
object_store: s3
schema: v13
index:
prefix: loki_index_
period: 24h
storage_config:
aws:
region: eu-west-1
bucketnames: chunks_bucket
s3forcepathstyle: true
storage:
type: s3
bucketNames:
chunks: "chunks_bucket"
ruler: "ruler_bucket"
s3:
region: eu-west-1
gateway:
basicAuth:
enabled: true
existingSecret: loki-basic-auth
My Grafana data source has the following configuration:
Authentication --> Basic authentication --> user = 123, password = <pwd>
HTTP Headers --> Header = X-Scope-OrgID, Values = abc
What could be wrong in my setup, so I can’t get the desired outcome of 2 folders inside the chunks_bucket with name corresponding to my tenant-ids (abc & xyz)