Multiple prometheus datasources

Grafana Prometheus
1

We’ve multiple set of prometheus instances(2 prometheus per set) running for monitoring different services. We’ve a shared grafana cluster (3 instances with a mysql db).

I am planning to set a loadbalancer infront of these prometheus & then use the loadbalancer as the datasource endpoint.

Will there be any issue when the requests (from grafana to prometheus) will be load balanced ?

PS : I tried to search related discussion but didn’t get any answer on this.
Also, new to grafana so not sure on how to use datasource template variable (as shared here Recommended HA setup)

Also, we;ve all the setup running on aws ec2.

2

Hi @karancode,

take a look at Promxy.

Promxy is a prometheus proxy that makes many shards of prometheus appear as a single API endpoint to the user.

We are in the process of rolling out Promxy for a very similar use case as the one you are describing. At the moment we are also testing putting HAProxy in front of Promxy. Not sure how useful that is yet.

Cheers,
b0b

3

@b0b
Thanks for the update. I had checked Promxy already & I think I would like to give it a try but as of now I wanted to check on other ways as well.
Manily I am trying to figure why a LoadBalancer won’t work.