Connection error for data source Prometheus with grafana using Docker windows

Hello,

I need help.
I install grafana and prometheus and when I want connect data source I have this error message “Error reading Prometheus: Post “http://192.168.x.133:9090/api/v1/query”: dial tcp 192.168.x.133:9090: connect: no route to host”

My file docker-compose.yml
version: “3.3”

services:
grafana:
image: grafana/grafana
container_name: grafana
ports:
- 3000:3000
volumes:
- ./data/grafana:/var/lib/grafana

prom:
image: prom/prometheus
container_name: prometheus
ports:
- 9090:9090
volumes:
- ./config/prometheus.yml:/etc/prometheus/prometheus.yml
- ./data/prometheus:/prometheus

node-exporter:
image: prom/node-exporter:latest
container_name: node-exporter
ports:
- 9100:9100

My file prometheus.yml
global:
scrape_interval: 15s

scrape_configs:

  • job_name: “prometheus”
    scrape_interval: 10s
    static_configs:

    • targets: [“192.168.x.133:9090”]
  • job_name: “node”
    scrape_interval: 10s
    static_configs:

    • targets: [“node-exporter:9100”]

please you can help me ?
Thanks :slight_smile:

Hi @escros,

Welcome to the :grafana: community forum !!

This seems to be a docker-compose configuration-specific case.

Can you please check if port 9090 is allowed in the firewall (if you are using one).

Another way to verify is to use the ss (or netstat) command e.g.

ss -tupln | grep 9090

Lastly, more information can be found in the command docker logs prometheus to see what is going on…

Hello @usmanahmad

Thanks for your answer.
Here is the netstat, I have the port but not assigned to my ip is this normal ?
Connexion actives

Proto Adresse locale Adresse distante État État de déchargement

TCP 0.0.0.0:88 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:135 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:389 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:445 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:464 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:593 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:636 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:2179 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:3000 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:3268 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:3269 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:5985 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:9090 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:9100 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:9182 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:9389 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:47001 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:49664 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:49665 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:49666 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:49668 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:49669 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:49670 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:49672 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:49673 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:49681 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:49691 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:49712 SRV-RSC:0 LISTENING InHost
TCP 0.0.0.0:54024 SRV-RSC:0 LISTENING InHost
TCP 127.0.0.1:53 SRV-RSC:0 LISTENING InHost
TCP 172.18.0.15:53 SRV-RSC:0 LISTENING InHost
TCP 172.18.0.15:139 SRV-RSC:0 LISTENING InHost
TCP 192.168.x.133:53 SRV-RSC:0 LISTENING InHost
TCP 192.168.x.133:139 SRV-RSC:0 LISTENING InHost
TCP 192.168.x.133:3000 host:55766 ESTABLISHED InHost
TCP 192.168.x.133:3000 host:59726 ESTABLISHED InHost
TCP 192.168.x.133:55766 host:3000 ESTABLISHED InHost
TCP 192.168.x.133:55920 host:3000 TIME_WAIT InHost
TCP 192.168.x.133:55921 host:3000 TIME_WAIT InHost
TCP 192.168.x.133:55922 host:3000 TIME_WAIT InHost
TCP 192.168.x.133:55923 host:3000 TIME_WAIT InHost
TCP 192.168.x.133:59726 host:3000 ESTABLISHED InHost

The logs Docker logs prometheus
ts=2022-08-08T14:09:59.740Z caller=main.go:491 level=info msg="No time or size retention was set so using the default time retention" duration=15d ts=2022-08-08T14:09:59.741Z caller=main.go:535 level=info msg="Starting Prometheus Server" mode=server version="(version=2.37.0, branch=HEAD, revision=b41e0750abf5cc18d8233161560731de05199330)" ts=2022-08-08T14:09:59.742Z caller=main.go:540 level=info build_context="(go=go1.18.4, user=root@0ebb6827e27f, date=20220714-15:13:18)" ts=2022-08-08T14:09:59.745Z caller=main.go:541 level=info host_details="(Linux 5.10.104-linuxkit #1 SMP Thu Mar 17 17:08:06 UTC 2022 x86_64 8fe5c97ac161 (none))" ts=2022-08-08T14:09:59.745Z caller=main.go:542 level=info fd_limits="(soft=1048576, hard=1048576)" ts=2022-08-08T14:09:59.745Z caller=main.go:543 level=info vm_limits="(soft=unlimited, hard=unlimited)" ts=2022-08-08T14:09:59.760Z caller=web.go:553 level=info component=web msg="Start listening for connections" address=0.0.0.0:9090 ts=2022-08-08T14:09:59.761Z caller=main.go:972 level=info msg="Starting TSDB ..." ts=2022-08-08T14:09:59.766Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false ts=2022-08-08T14:09:59.778Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1659955302240 maxt=1659960000000 ulid=01G9YV1878YTAQAMQYPGQPM7HJ ts=2022-08-08T14:09:59.874Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" ts=2022-08-08T14:09:59.893Z caller=head.go:536 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=18.4129ms ts=2022-08-08T14:09:59.893Z caller=head.go:542 level=info component=tsdb msg="Replaying WAL, this may take a while" ts=2022-08-08T14:09:59.980Z caller=head.go:578 level=info component=tsdb msg="WAL checkpoint loaded" ts=2022-08-08T14:09:59.990Z caller=head.go:613 level=info component=tsdb msg="WAL segment loaded" segment=6 maxSegment=13 ts=2022-08-08T14:10:00.008Z caller=head.go:613 level=info component=tsdb msg="WAL segment loaded" segment=7 maxSegment=13 ts=2022-08-08T14:10:00.067Z caller=head.go:613 level=info component=tsdb msg="WAL segment loaded" segment=8 maxSegment=13 ts=2022-08-08T14:10:00.090Z caller=head.go:613 level=info component=tsdb msg="WAL segment loaded" segment=9 maxSegment=13 ts=2022-08-08T14:10:00.135Z caller=head.go:613 level=info component=tsdb msg="WAL segment loaded" segment=10 maxSegment=13 ts=2022-08-08T14:10:00.150Z caller=head.go:613 level=info component=tsdb msg="WAL segment loaded" segment=11 maxSegment=13 ts=2022-08-08T14:10:00.179Z caller=head.go:613 level=info component=tsdb msg="WAL segment loaded" segment=12 maxSegment=13 ts=2022-08-08T14:10:00.184Z caller=head.go:613 level=info component=tsdb msg="WAL segment loaded" segment=13 maxSegment=13 ts=2022-08-08T14:10:00.186Z caller=head.go:619 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=87.5684ms wal_replay_duration=205.3823ms total_replay_duration=311.4043ms ts=2022-08-08T14:10:00.192Z caller=main.go:993 level=info fs_type=65735546 ts=2022-08-08T14:10:00.192Z caller=main.go:996 level=info msg="TSDB started" ts=2022-08-08T14:10:00.192Z caller=main.go:1177 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml ts=2022-08-08T14:10:00.199Z caller=main.go:1214 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=6.5438ms db_storage=1.6µs remote_storage=1.1µs web_handler=500ns query_engine=1.4µs scrape=1.0634ms scrape_sd=27.4µs notify=700ns notify_sd=1.1µs rules=1.1µs tracing=3.4µs ts=2022-08-08T14:10:00.200Z caller=main.go:957 level=info msg="Server is ready to receive web requests." ts=2022-08-08T14:10:00.200Z caller=manager.go:941 level=info component="rule manager" msg="Starting rule manager..." ts=2022-08-08T15:00:01.276Z caller=compact.go:519 level=info component=tsdb msg="write block" mint=1659960000000 maxt=1659967200000 ulid=01G9YZGK8PDX5XM5B6X95AZ9D0 duration=357.7604ms ts=2022-08-08T15:00:01.316Z caller=head.go:842 level=info component=tsdb msg="Head GC completed" duration=5.4372ms ts=2022-08-08T15:00:01.335Z caller=checkpoint.go:97 level=info component=tsdb msg="Creating checkpoint" from_segment=6 to_segment=10 mint=1659967200000 ts=2022-08-08T15:00:01.777Z caller=head.go:1011 level=info component=tsdb msg="WAL checkpoint complete" first=6 last=10 duration=447.8679ms

Thanks. As I did not see any error in the logs so I wanted to try out your provided code on my machine (as a tip, please use the code highlighter when pasting code as it keeps the YAML syntax and indentation in tact :slight_smile: )

Here is your docker-compose.yml file. I had to fix some parts as it was not running correctly. I added comments where I had made changes.

version: "3"

services:
  grafana:
    image: grafana/grafana
    container_name: grafana
    
    # Required valid permission for /var/lib/grafana to create files and folders
    # Since I run from the root so both uid and gid are 0:0 
    # but you can use the command i.e. (id) without brackets to find your user correct numbers
    user: "0:0"
    ports:
      - 3000:3000
    volumes:
      - ./data/grafana:/var/lib/grafana

  prom:
    image: prom/prometheus
    container_name: prometheus
    ports:
      - 9090:9090
    volumes:
      - ./config/prometheus.yml:/etc/prometheus/prometheus.yml
     
     # fixed the path here as originally you wrote without /etc
     # while in above line you are mentioning /etc/ so was causing errors
      - ./data/prometheus:/etc/prometheus

  node-exporter:
    image: prom/node-exporter:latest
    container_name: node-exporter
    ports:
      - 9100:9100

Here is your prometheus.yml file inside the directory config which was mentioned in the docker-compose.yml file:

global:
  scrape_interval: 15s

scrape_configs:

  - job_name: "prometheus"
    scrape_interval: 10s

    static_configs:

      - targets: ["192.168.122.169:9090"]

  - job_name: "node"
    scrape_interval: 10s

    static_configs:

      - targets: ["192.168.122.57:9100"]
~                                            

If you do docker-compse up -d it will run correctly.

Now coming to the point of;

http://192.168.122.169:9090/api/v1/query

So to my best understanding, this is basically a way to call the Prometheus HTTP API.

But it needs parameters e.g. GET or POST along with something to query.

According to the original documentation;

You can easily test and verify via the cURL command on your machine e.g.

[root@docker test]# curl 'http://192.168.122.169:9090/api/v1/query?query=up&time=2015-07-01T20:10:51.781Z'

{"status":"success","data":{"resultType":"vector","result":[]}}

Finally, you can try out the same thing on the browser by just pasting the URL part and it should show the same result

Hello @usmanahmad
Thanks for answer but I still have the same problem " Get “http://192.168.194.133:9100/metrics”: dial tcp 192.168.194.133:9100: connect: no route to host"
I think there is something missing in the docker configuration with the permissions?
I am on windows

I think this is definitely something related to the configuration setting on Windows.

I am not sure but maybe check using the simple test for e.g. running Nginx image on port 80 and see if you can access it from your browser? if port 80 works, then create the same image but on port 3001 to see if you can access non-regular ports.

2nd is to check any firewall (either Windows Firewall service or any custom Firewall software) which might be blocking the ports.

Thank for your help :slight_smile:
I uninstall and reinstall docker and that works

1 Like