I cannot load custom grafana.ini config file in docker-compose

Grafana Configuration
,
1

  • What Grafana version and what operating system are you using?
    grafana/grafana 9.3.2

  • What are you trying to achieve?
    Run Grafana with my own grafana.ini configuration.

  • How are you trying to achieve it?
    I run Grafana from docker-compose.

  • What happened?
    Grafana won’t to run.

  • What did you expect to happen?
    Grafana will start with my configuration file.

  • Can you copy/paste the configuration(s) that you are having problems with?
    docker-compose.yml:

  grafana:
    image: grafana/grafana:9.3.2
    container_name: grafana
    volumes:
      - ./__DATA__/grafana_data:/var/lib/grafana:rw
      - ./__DATA__/grafana_etc:/etc/grafana:rw
    environment:
      - GF_SECURITY_ADMIN_USER=${ADMIN_USER:-admin}
      - GF_SECURITY_ADMIN_PASSWORD=${ADMIN_PASSWORD:-admin}
      - GF_PATHS_CONFIG="/etc/grafana/grafana.ini"
    restart: unless-stopped
    expose:
      - 4000
    network_mode: host
    user: "472"
    labels:
      org.label-schema.group: "monitoring"

Directories that are mounted:

$ ls -la __DATA__/grafana_etc/
total 64
drwxr-xr-x 3 472 root  4096 Feb  8 15:58 .
drwxr-xr-x 7 pi  pi    4096 Feb  8 15:52 ..
-rw-r--r-- 1 472 root 51569 Feb  8 15:58 grafana.ini
drwxr-xr-x 4 472 root  4096 Jan  4 22:29 provisioning
$
$
$ ls -la __DATA__/grafana_data/
total 1476
drwxr-xr-x 7 472 root    4096 Feb  8 15:47 .
drwxr-xr-x 7 pi  pi      4096 Feb  8 15:52 ..
drwxr-x--- 3 472 root    4096 Jan  4 23:22 alerting
drwx------ 2 472 root    4096 Jan  4 23:22 csv
drwxr-x--- 2 472 root    4096 Jan  4 23:22 file-collections
-rw-r----- 1 472 root 1425408 Feb  8 15:47 grafana.db
-rw-r--r-- 1 472 root   51569 Feb  8 08:31 grafana.ini
drwxr-xr-x 2 472 root    4096 Jan  4 23:22 plugins
drwx------ 2 472 root    4096 Jan  4 23:22 png

Running process in container:

 $ docker exec -ti grafana bash -c "ps aux | cat"
PID   USER     TIME  COMMAND
    1 grafana   0:00 grafana-server --homepath=/usr/share/grafana --config="/etc/grafana/grafana.ini" --packaging=docker cfg:default.log.mode=console cfg:default.paths.data=/var/lib/grafana cfg:default.paths.logs=/var/log/grafana cfg:default.paths.plugins=/var/lib/grafana/plugins cfg:default.paths.provisioning=/etc/grafana/provisioning

Envs from container:

$ docker exec -ti grafana env
PATH=/usr/share/grafana/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=malina
TERM=xterm
GF_SECURITY_ADMIN_USER=admin
GF_SECURITY_ADMIN_PASSWORD=admin
GF_PATHS_CONFIG="/etc/grafana/grafana.ini"
GF_PATHS_DATA=/var/lib/grafana
GF_PATHS_HOME=/usr/share/grafana
GF_PATHS_LOGS=/var/log/grafana
GF_PATHS_PLUGINS=/var/lib/grafana/plugins
GF_PATHS_PROVISIONING=/etc/grafana/provisioning
HOME=/home/grafana
  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
    This is log from Grafana:
 $ docker-compose logs -f grafana
Attaching to grafana
grafana         | GF_PATHS_CONFIG='"/etc/grafana/grafana.ini"' is not readable.
grafana         | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later
grafana         | logger=settings t=2023-02-08T15:12:32.671750491Z level=error msg="failed to parse \"\\\"/etc/grafana/grafana.ini\\\"\": open \"/etc/grafana/grafana.ini\": no such file or directory"
grafana         | GF_PATHS_CONFIG='"/etc/grafana/grafana.ini"' is not readable.
grafana         | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later
grafana         | logger=settings t=2023-02-08T15:12:33.737623716Z level=error msg="failed to parse \"\\\"/etc/grafana/grafana.ini\\\"\": open \"/etc/grafana/grafana.ini\": no such file or directory"
grafana         | GF_PATHS_CONFIG='"/etc/grafana/grafana.ini"' is not readable.

This is list of files from /etc/grafana from the container:

$ docker exec -ti grafana ls -l /etc/grafana/
total 56
-rw-r--r--    1 grafana  root         51569 Feb  8 14:58 grafana.ini
drwxr-xr-x    4 grafana  root          4096 Jan  4 21:29 provisioning

Mounts in container:

$ docker exec -ti grafana mount
overlay on / type overlay (rw,relatime,lowerdir=/var/lib/docker/overlay2/l/JEMVC4ZRIELGYUC4EFL57A743A:/var/lib/docker/overlay2/l/D5ER4XSHFYHP2ITR2CWABF64XO:/var/lib/docker/overlay2/l/HGO3G3DNWGXMZJNR4GCEL54ZDQ:/var/lib/docker/overlay2/l/K6JU5TKLBU67GFCTQQV4NNU7XQ:/var/lib/docker/overlay2/l/YYEXP363DUGSGGVBCTWKPTOCTY:/var/lib/docker/overlay2/l/4RDSQWDDKHVELXVNKEJVAKYMLX:/var/lib/docker/overlay2/l/YQLCVDF7TK2SHUQW6RKOLFUP2Y:/var/lib/docker/overlay2/l/RBXYAYRRLHZUJNWTACS237NGXZ:/var/lib/docker/overlay2/l/3AXPONQGHVSWGB7NA625J6BGFH:/var/lib/docker/overlay2/l/BXVLB4PUOCEKOBWJET4LCLCWTD,upperdir=/var/lib/docker/overlay2/544017f73decaf5aeb1a3df9ff06d303dc0550d6eae07184545ba019af904ded/diff,workdir=/var/lib/docker/overlay2/544017f73decaf5aeb1a3df9ff06d303dc0550d6eae07184545ba019af904ded/work)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev type tmpfs (rw,nosuid,size=65536k,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666)
sysfs on /sys type sysfs (ro,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup type cgroup2 (ro,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
/dev/root on /etc/grafana type ext4 (rw,noatime,stripe=8191)
/dev/root on /etc/resolv.conf type ext4 (rw,noatime,stripe=8191)
/dev/root on /etc/hostname type ext4 (rw,noatime,stripe=8191)
/dev/root on /etc/hosts type ext4 (rw,noatime,stripe=8191)
shm on /dev/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=65536k)
/dev/root on /var/lib/grafana type ext4 (rw,noatime,stripe=8191)
proc on /proc/bus type proc (ro,relatime)
proc on /proc/fs type proc (ro,relatime)
proc on /proc/irq type proc (ro,relatime)
proc on /proc/sys type proc (ro,relatime)
proc on /proc/sysrq-trigger type proc (ro,relatime)
tmpfs on /proc/asound type tmpfs (ro,relatime)
tmpfs on /proc/keys type tmpfs (rw,nosuid,size=65536k,mode=755)
tmpfs on /proc/latency_stats type tmpfs (rw,nosuid,size=65536k,mode=755)
tmpfs on /proc/timer_list type tmpfs (rw,nosuid,size=65536k,mode=755)
tmpfs on /sys/firmware type tmpfs (ro,relatime)
  • Did you follow any online instructions? If so, what is the URL?
    • I opened URL from log but I think my setting is alright.
    • I spent at least half day with reading different configuration, docker-compose.yamls, forums, … and I didn’t find out where I did a mistake.
    • I also tried to mount grafana.ini as /grafana.ini but it finished with the same result.

EDIT: I added more debug information.

2

Originally I changed grafana.ini file according to my needs and I tried to load it. But it didn’t work. So I have changed my approach to the issue. Instead of editing grafana.ini I set new ENV variables and left grafana.ini as is. And voila it works!

One thing that I learned and consider to be important is: Any config from the config file can be overridden by the env variable. The syntax for ENV variable name: GF_<CONF-SECTION>_<CONFIG-PROPERTY>.

If you have in your config file section:

[server]
http_port = 4000

So equivalent env variable is:

GF_SERVER_HTTP_PORT=4000

That’s a brilliant idea from Grafana Labs.

Just for complete information, I put there my docker-compose.yaml:

  grafana:
    image: grafana/grafana:9.3.2
    container_name: grafana
    volumes:
      - ./__DATA__/grafana_data:/var/lib/grafana:rw
      - ./grafana/provisioning/dashboards:/etc/grafana/provisioning/dashboards
      - ./grafana/provisioning/datasources:/etc/grafana/provisioning/datasources
    environment:
      - GF_SECURITY_ADMIN_USER=${ADMIN_USER:-admin}
      - GF_SECURITY_ADMIN_PASSWORD=${ADMIN_PASSWORD:-admin}
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Editor
      - GF_SERVER_HTTP_PORT=4000
      - GF_USERS_ALLOW_SIGN_UP=false
    restart: unless-stopped
    expose:
      - 4000
    network_mode: host
    user: "472"
    labels:
      org.label-schema.group: "monitoring"
2 Likes
3

Thanks for sharing your solution.

I got same problem, as using a terminal in a docker, I don’t have access to edit the config file.\

Updating ENV variants works for me.

1 Like