How to pass tenant name in the RemoteWrite

Grafana
1

Hi,

I am using Prometheus, Grafana and Mimir stack for monitorting the k8s cluster, I am using following helm charts:

  • Promethus: https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack
  • Grafana: https://github.com/grafana/helm-charts/tree/main/charts/grafana
  • Mimir: https://github.com/grafana/mimir/tree/main/operations/helm/charts/mimir-distributed

In the prometheus config, I am trying to send the tenant name in the header, following the is values.yaml for prometheus helm chart:

prometheus:
  agentMode: false
  prometheusSpec:
    podMonitorSelectorNilUsesHelmValues: false
    serviceMonitorSelectorNilUsesHelmValues: false
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
              - key: "karpenter.sh/capacity-type"
                operator: In
                values:
                  - spot
    remoteWrite:
      - url: http://mimir-nginx.monitoring.svc:80/api/v1/push
        headers:
          X-Scope-OrgID: "tenant1"
    remoteRead:
      - url: http://mimir-nginx.monitoring.svc:80/api/v1/read

This was referred from doc: prometheus-operator/Documentation/api.md at main · prometheus-operator/prometheus-operator · GitHub

But after adding the header in the Prometheus, I do not see any metircs/data in Grafana, and nor errors on the Prometheus either. Anything wrong I am doing here?

Thanks in advance