I am trying to set up tempo in an AKS cluster using this values.yaml with federated token
tempo:
mode: distributed
storage:
trace:
blocklist_poll_tenant_index_builders: 1
blocklist_poll_jitter_ms: 500
backend: azure
blocklist_poll: 5m
azure:
container_name: <container-name>
storage_account_name: <storage-account-name>
use_federated_token: true
pool:
max_workers: 400
queue_depth: 20000
search:
prefetch_trace_count: 1000
wal:
path: /var/tempo/wal
serviceAccount:
name: tempo
annotations:
azure.workload.identity/client-id: ""
labels:
azure.workload.identity/use: "true"
image:
tag: "2.8.1"
searchEnabled: true
compactor:
extraArgs:
- -config.expand-env=true
extraEnv:
- name: AZURE_CLIENT_ID
value: ""
ingester:
extraArgs:
- -config.expand-env=true
replicaCount: 3
extraEnv:
- name: AZURE_CLIENT_ID
value: ""
querier:
extraArgs:
- -config.expand-env=true
replicaCount: 2
extraEnv:
- name: AZURE_CLIENT_ID
value: ""
memberlist:
join_members:
- tempo-gossip-ring:7946
traces:
otlp:
grpc:
enabled: true
http:
enabled: true
distributor:
log_received_spans:
enabled: true
log_discarded_spans:
enabled: true
extraArgs:
- -config.expand-env=true
receivers:
otlp:
protocols:
grpc: 0.0.0.0:4317
http: 0.0.0.0:4318
extraEnv:
- name: AZURE_CLIENT_ID
value: ""
service:
type: LoadBalancer
ports:
- name: otlp-grpc
port: 4317
targetPort: 4317
- name: otlp-http
port: 4318
targetPort: 4318
- name: http-metrics
port: 3200
targetPort: 3200
- name: grpc-metrics
port: 9095
targetPort: 9095
queryFrontend:
extraArgs:
- -config.expand-env=true
service:
type: LoadBalancer
ports:
- name: http-query
port: 3200
targetPort: 3200
- name: grpc-query
port: 9095
targetPort: 9095
extraEnv:
- name: AZURE_CLIENT_ID
value: ""
I am able to connect to grafana and send traces to tempo but they only stay for a bit. So when i checked the config map it showed the storage backend to be local instead of azure
storage:
trace:
backend: local
blocklist_poll: 5m
local:
path: /var/tempo/traces
pool:
max_workers: 400
queue_depth: 20000
search:
prefetch_trace_count: 1000
wal:
path: /var/tempo/wal
Tried editing the config map from local to azure, then the pods starts crashing with this error
level=error ts=2025-08-18T10:12:26.996639065Z caller=main.go:121 msg="error running Tempo" err="failed to init module services: error initialising module: store: failed to create store: getting storage container: no client ID specified. Check pod configuration or set ClientID in the options".
what is the problem here?