Host not found in upstream "grafana" error

I encountered an issue today with the Nginx service running as a container on our Ubuntu 20.04.6 LTS server. The service was in a stopped state, and I had to restart it manually.

I’ve attached a snippet from the syslog for reference. It appears that several services were restarted automatically, but Nginx failed to start.

2024/02/28 06:44:44 [notice] 1#1: signal 3 (SIGQUIT) received, shutting down
2024/02/28 06:44:44 [notice] 21#21: gracefully shutting down
2024/02/28 06:44:44 [notice] 21#21: exiting
2024/02/28 06:44:44 [notice] 21#21: exit
2024/02/28 06:44:44 [notice] 1#1: signal 17 (SIGCHLD) received from 21
2024/02/28 06:44:44 [notice] 1#1: worker process 21 exited with code 0
2024/02/28 06:44:44 [notice] 1#1: exit
2024/02/28 06:44:47 [emerg] 1#1: host not found in upstream "grafana" in /etc/nginx/conf.d/grafana.conf:15
nginx: [emerg] host not found in upstream "grafana" in /etc/nginx/conf.d/grafana.conf:15
2024/02/28 06:44:48 [emerg] 1#1: host not found in upstream "grafana" in /etc/nginx/conf.d/grafana.conf:15
nginx: [emerg] host not found in upstream "grafana" in /etc/nginx/conf.d/grafana.conf:15

I have both Grafana and Nginx running as Docker images on my server, with Nginx serving as a reverse proxy. However, I faced a problem when the system stopped the Nginx service during daily cleanup activities. Unfortunately, the service failed to restart, and upon checking the Docker logs for Nginx, I found the above error. Upon further investigation, it seems that the upstream block is not present in our Nginx configuration file located at /etc/nginx/conf.d/grafana.conf . Currently, we only have “grafana” specified as the backend service.

  1. Is it necessary to add the upstream block in our Nginx configuration?
  2. How can we prevent similar errors from occurring in the future?

I appreciate your assistance in resolving this matter promptly.

Regards,
Alakananda S