Grafana does not send notifications to Telegram through reverse proxy

Grafana Alerting
1

Hello,

I am using Grafana on a server that does not have Internet, I need to send Telegram notifications, for that I am using Nginx server in another machine.

Server A:

  • Grafana installed / Running on port 3000.
  • IP of the server (example): 10.16.216.122
  • I also configured /etc/hosts to send that notifications to my server:
#/etc/hosts
10.16.216.16  api.telegram.org

Server B:

  • IP of the server (example): 10.16.216.16
  • Nginx installed, with this config:
server {
    listen 80;
    listen [::]:80 http2 ssl;

    server_name http://10.16.216.122;
    location / {
        proxy_pass http://10.16.216.122:3000/;
    }}

When I try to send the notification, I receive this in the logs:

t=2021-10-27T17:10:07+0200 lvl=eror msg="Failed to send webhook" logger=alerting.notifier.telegram error="Post https://api.telegram.org/botXXXXXXXXXX/sendMessage: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" webhook=TelegramTest
t=2021-10-27T17:10:07+0200 lvl=eror msg="failed to send notification" logger=alerting.notifier uid= error="Post https://api.telegram.org/botXXXXXXXXXXXXXX/sendMessage: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
t=2021-10-27T17:10:07+0200 lvl=eror msg="failed to send notification" logger=alerting.notifier uid= error="Post https://api.telegram.org/botXXXXXXXXXXXXXXXXXXX/sendMessage: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
t=2021-10-27T17:10:07+0200 lvl=eror msg="Failed to send alert notifications" logger=context userId=51 orgId=5 uname=XXXX error="Post https://api.telegram.org/botXXXXXXXXXXXXX/sendMessage: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
t=2021-10-27T17:10:07+0200 lvl=eror msg="Request Completed" logger=context userId=51 orgId=5 uname=XXXXX method=POST path=/api/alert-notifications/test status=500 remote_addr=10.26.216.122 time_ms=30006 size=48 referer=http://10.26.216.122/alerting/notification/1/edit

Am I missing something in my configuration on Nginx? Does the Server A need something more to work?

2

It looks like nginx may not be configured correctly. Are you able to curl Server B from Server A?

3

This topic was automatically closed after 365 days. New replies are no longer allowed.