We are deploying Grafana using a helm chart into EKS. Everything seems to be working and have OpsGenie set as a contact point so that alerts are routed to our on-call resource.
I am trying to set up Email as a secondary contact point using smtp configuration to an internal postfix relay, which is also running in the same EKS cluster. I added the following smtp configuration has been added to the grafana.ini file and restarted the grafana pods, but I am unable to send a test email through the Grafana UI.
[smtp] enabled = true host = relayer.postfix.svc.cluster.local:25 from_address = no-reply@email.com skip_verify = true
The postfix server does not require authentication. When I try sending a test email, the call will eventually time out with a 504 Gateway Timeout error. I have debug logging enabled, but it doesn’t appear that this connection attempt is getting logged. Similarly, not finding much in the postfix logs either.
I’ve been able to test the same smtp configuration using telnet from a debug pod deployed to the same namespace and successfully sent/received an email from there. I also updated the smtp configuration to use a gmail account (smtp.gmail.com:25) and was successfully able to receive an email through there as well.
I found a comment here from a year ago that seems to suggest that TLS is required unless the smtp server is also on localhost. I’m not 100% sure this is the issue, but it is the closest thing I’ve been able to find so far: https://github.com/grafana/grafana/issues/69654#issuecomment-1584306140
skip_verify is set to true, but I’m not sure if that disables TLS or just the verification. Can anyone confirm if TLS is required for this scenario or have a suggestion on how to resolve this?
Thanks