Grafana behind Traefik reverse-proxy in Docker Swarm

Grafana Configuration
,
1

Hey guys. I am pretty sure there are several of these questions all over the internet. However I could no find any one solution. I am running Docker Swarm, and it has the following configuration.

I have many other services running similarly and they are all being proxied successfully by Traefik. Does Grafana have any special config besides the ones described here?

In the logs of grafana I am not getting any errors, but I am getting this info on the http server

image
image872×104 20.1 KB

With this configuration I am getting a blank page stating Internal Server Error.

version: '3.8'

networks:
  proxy:
    external: true

services:

  traefik:
    image: 'traefik'
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik_logs:/var/log
    networks:
      - proxy
    command:
      - --api
      - --ping  # enable healthchecking
      - --providers.docker=true
      - --providers.docker.exposedbydefault=false
      - --providers.docker.network=proxy
      - --providers.docker.swarmMode=true

      - --serversTransport.insecureSkipVerify=true  # just for portainer

      - --entrypoints.web.address=:80
      - --entrypoints.web.http.redirections.entryPoint.to=websecure
      - --entrypoints.web.http.redirections.entryPoint.scheme=https
      - --entrypoints.websecure.address=:443

      - --log=true
      - --log.format=json
      - --log.level=DEBUG
      - --log.filepath=/var/log/traefik.log

      - --accesslog=true
      - --accesslog.filepath=/var/log/access.log
      - --accesslog.format=json

      - --metrics.prometheus=true
      - --entryPoints.metrics.address=:8082
      - --metrics.prometheus.entryPoint=metrics

      - --tracing.jaeger=true

    ports:
      - 80:80    # http
      - 443:443  # https entrypoint
    deploy:
      labels:
        traefik.enable: 'true'
        traefik.http.routers.api.rule: Host(`traefik.${DOMAIN}`)
        traefik.http.routers.api.service: api@internal
        traefik.http.routers.api.tls: 'true'
        # swarm mode discovery
        traefik.http.services.traefik.loadbalancer.server.port: 80
      placement:
        constraints:
          - node.role == manager

  grafana:
    # https://grafana.com/docs/grafana/latest/administration/configure-docker/
    image: grafana/grafana-oss
    networks:
      - proxy
    ports:
      - '3000:3000'
    # https://grafana.com/docs/grafana/next/setup-grafana/configure-grafana/#override-configuration-with-environment-variables
    environment:
      GF_INSTALL_PLUGINS: "grafana-clock-panel,grafana-simple-json-datasource"
      GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_SECURITY_ADMIN_PASSWORD}
      GF_SERVER_DOMAIN: grafana.${DOMAIN}
      GF_SERVER_ROOT_URL: https://grafana.${DOMAIN}
      GF_USERS_ALLOW_SIGN_UP: 'false'
    deploy:
      labels:
        traefik.enable: 'true'
        traefik.http.routers.grafana_rtr.rule: Host(`grafana.${DOMAIN}`)
        traefik.http.routers.grafana_rtr.tls: 'true'
        traefik.http.routers.grafana_rtr.service: grafana
        traefik.http.services.grafana.loadbalancer.server.port: 3000
        traefik.http.services.grafana.loadbalancer.server.scheme: https
        traefik.docker.network: proxy
2

I found the culprit. Removing the label:

traefik.http.services.grafana.loadbalancer.server.scheme: https

Made it work. However, I think that line is important to make other stuff work in the proxy, likely the Let’s Encrypt certificates