Grafana, Prometheus and VPN


I’ve got a machine on AWS that should be monitoring another machine on-prem.
I use Zabbix and Grafana for all my monitoring needs.
Zabbix-Agent takes the metrics all good, and no issues there.
However, Grafana cannot get metrics.

My setup (high level):

[ OnPrem Minikube ] <— AWS IPSEC —> [ AWS VPC [Grafana] - [Zabbix] ]

OnPrem minikube has connectivity via NAT for updates and other stuff.
Cannot open ports on the OnPrem network, so having Grafana directly available over the internet is not possible.

If i curl -fsSL http://:30000/metrics from the Grafana machine on AWS i can get all the Prometheus metrics.

If i add the address to the Grafana Prometheus source, shows “Datasource updated”, and does not show success message.
If i try querying, no results are visible (time-out).

Any ideas on how this can work?
Thanks in advance!

If you only have http/https in the NAT connection, just add nginx/httpd on premmise with reverse proxy that pointed to the OnPrem Minikube, then config the prometheus scrape target into that nginx reverse proxy location that interested.

Thanks, will try that setup.