Hi team,
Please advise on my issue with Grafana mimir. I cannot find data in minio pod after enabling remote-write in prometheus
- What Grafana version and what operating system are you using?
Rancher Kubernetes cluster with latest Grafana version - What are you trying to achieve?
seeing data in /data in minio pod - How are you trying to achieve it?
I am using Helm to set up Prometheus and Grafana Mimir. My goal is to confirm that Prometheus data appears in the MinIO pod. However, I cannot see any data under the /data folder.
My current settings are:
Prometheus installation:
helm install prometheus prometheus-community/prometheus -n monitoring
- adding remote write into the configmap of prometheus server:
prometheus.yml: |
global:
evaluation_interval: 1m
scrape_interval: 1m
scrape_timeout: 10s
rule_files:
- /etc/config/recording_rules.yml
- /etc/config/alerting_rules.yml
- /etc/config/rules
- /etc/config/alerts
remote_write:
- url: http://mimir-nginx.mimir-test.svc:80/api/v1/push
scrape_configs:
- job_name: prometheus
static_configs:
Mimir installation:
helm install mimir grafana/mimir-distributed -n monitoring
- What happened?
Cannot see the prometheus data in minio pod:
- Enter minio pod using kubectl exec command
user@DKT:/mnt/c/Users/user$ k exec -it mimir-minio-7ffd96b679-6wb9k -n mimir-test -- sh sh-4.4$ sh-4.4$ cd /data sh-4.4$ ls sh-4.4$ ls -lrt
-
What did you expect to happen?
Prometheus data to be appeared under /data in minio pod -
Can you copy/paste the configuration(s) that you are having problems with?
-
Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
No -
Did you follow any online instructions? If so, what is the URL?
Get started with Grafana Mimir using the Helm chart | Grafana Labs Helm charts documentation
I am not sure if I missed anything. I tested prometheus/mimir combo successfully using docker-compose but having issue with the Helm setup in Kubernetes cluster.
Please kindly help. Many thanks.