Where is this port 53 coming from?

Wondering if anyone has encountered this error before:

Post “http://lx-4:9095/api/v1/query”: dial tcp: lookup lx-4 on 138.153.159.32:53: server misbehaving - There was an error returned querying the Prometheus API.

  • What Grafana version and what operating system are you using?

Latest version and Linux.

  • What are you trying to achieve?

To connect my Prometheus server / data source to my Grafana server.

  • How are you trying to achieve it?

By filling out the add data source connection form in the UI and clicking “Save & Test” at the bottom.

  • What happened?

I get error: There was an error returned querying the Prometheus API

  • What did you expect to happen?

The connection to be successful.

  • Can you copy/paste the configuration(s) that you are having problems with?

global:
scrape_interval: 15s

scrape_configs:

  • job_name: ‘prometheus’
    static_configs:
    • targets: [‘lx-4:9095’]
  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.

Post “http://lx-4:9095/api/v1/query”: dial tcp: lookup lx-4 on 138.153.159.32:53: server misbehaving - There was an error returned querying the Prometheus API.

  • Did you follow any online instructions? If so, what is the URL?

Port 53 is for DNS usually. So your error message indicates that there was a problem to resolve record lx-4 by using DNS server 138.153.159.32. So make sure youu have correct DNS configuration on the machine where this Prometheus is running.

Sorry I forgot to mention I am running both Grafana and Prometheus as Docker containers.

It doesn’t matter. You(Prometheus) wants to connect to some DNS name - e.g. www.google.com (in your case lx-4), so system must know IP address where to connect. If you are using Docker, then is Docker documentation is the right doc for this use case.

But isn’t lx-4 and it’s IP address used interchangeably in this context?