Grafana docker upgrade fails

  • What Grafana version and what operating system are you using?
    Grafana 9.0.1 , Ubuntu 20.0.4

  • What are you trying to achieve?
    Upgrade the docker container to latest version

  • How are you trying to achieve it?
    By deleting existing container and image and loading new image using following command

echo "Container upgrade started.."
sudo docker stop grafana
echo "Container Stopped."
sudo docker rm `sudo docker ps -aqf "name=grafana"`
echo "Container Deleted."
sudo docker rmi `sudo docker images grafana/grafana -q`
echo "Container Image Deleted."


docker run -d \
        --name=grafana \
        --restart unless-stopped \
        -p 3000:3000 \
        -e "GF_INSTALL_PLUGINS=grafana-clock-panel,grafana-simple-json-datasource,alexanderzobnin-zabbix-app,marcusolsson-dynamictext-panel" \
        -e "GF_LOG_MODE=console file" \
        -e TZ=Asia/Kolkata \
        -v grafana-data:/var/lib/grafana \
        -v grafana-config:/etc/grafana \
        -v grafana-def-config:/usr/share/grafana/conf \
        grafana/grafana:latest

echo "Latest Container Loaded"

docker ps -aqf "name=grafana"

  • What happened?
    When the container starts after the upgrade grafana keeps restarting. Please see log below. It shows no errors but just keeps restarting
logger=settings t=2022-06-24T11:57:33.349124489+05:30 level=info msg="Starting Grafana" version=9.0.1 commit=14e988bd22 branch=HEAD compiled=2022-06-21T19:13:01+05:30
logger=settings t=2022-06-24T11:57:33.350275082+05:30 level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini
logger=settings t=2022-06-24T11:57:33.350410172+05:30 level=info msg="Config loaded from" file=/etc/grafana/grafana.ini
logger=settings t=2022-06-24T11:57:33.350476949+05:30 level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana"
logger=settings t=2022-06-24T11:57:33.350574429+05:30 level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana"
logger=settings t=2022-06-24T11:57:33.350633483+05:30 level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins"
logger=settings t=2022-06-24T11:57:33.350693316+05:30 level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning"
logger=settings t=2022-06-24T11:57:33.350751944+05:30 level=info msg="Config overridden from command line" arg="default.log.mode=console"
logger=settings t=2022-06-24T11:57:33.350825665+05:30 level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana"
logger=settings t=2022-06-24T11:57:33.350884942+05:30 level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana"
logger=settings t=2022-06-24T11:57:33.350943386+05:30 level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins"
logger=settings t=2022-06-24T11:57:33.351029273+05:30 level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning"
logger=settings t=2022-06-24T11:57:33.351096846+05:30 level=info msg="Config overridden from Environment variable" var="GF_LOG_MODE=console file"
logger=settings t=2022-06-24T11:57:33.351154512+05:30 level=info msg="Path Home" path=/usr/share/grafana
logger=settings t=2022-06-24T11:57:33.351214881+05:30 level=info msg="Path Data" path=/var/lib/grafana
logger=settings t=2022-06-24T11:57:33.351271843+05:30 level=info msg="Path Logs" path=/var/log/grafana
logger=settings t=2022-06-24T11:57:33.351332305+05:30 level=info msg="Path Plugins" path=/var/lib/grafana/plugins
logger=settings t=2022-06-24T11:57:33.351387823+05:30 level=info msg="Path Provisioning" path=/etc/grafana/provisioning
logger=settings t=2022-06-24T11:57:33.351468192+05:30 level=info msg="App mode production"
logger=settings t=2022-06-24T11:57:58.366188735+05:30 level=info msg="Starting Grafana" version=9.0.1 commit=14e988bd22 branch=HEAD compiled=2022-06-21T19:13:01+05:30
logger=settings t=2022-06-24T11:57:58.367202589+05:30 level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini
logger=settings t=2022-06-24T11:57:58.36731531+05:30 level=info msg="Config loaded from" file=/etc/grafana/grafana.ini
logger=settings t=2022-06-24T11:57:58.367369512+05:30 level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana"
logger=settings t=2022-06-24T11:57:58.367431011+05:30 level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana"
logger=settings t=2022-06-24T11:57:58.367482899+05:30 level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins"
logger=settings t=2022-06-24T11:57:58.367534158+05:30 level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning"
logger=settings t=2022-06-24T11:57:58.367597842+05:30 level=info msg="Config overridden from command line" arg="default.log.mode=console"
logger=settings t=2022-06-24T11:57:58.367651674+05:30 level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana"
logger=settings t=2022-06-24T11:57:58.367703618+05:30 level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana"
logger=settings t=2022-06-24T11:57:58.367754839+05:30 level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins"
logger=settings t=2022-06-24T11:57:58.367818283+05:30 level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning"
logger=settings t=2022-06-24T11:57:58.367876078+05:30 level=info msg="Config overridden from Environment variable" var="GF_LOG_MODE=console file"
logger=settings t=2022-06-24T11:57:58.367939355+05:30 level=info msg="Path Home" path=/usr/share/grafana
logger=settings t=2022-06-24T11:57:58.367994021+05:30 level=info msg="Path Data" path=/var/lib/grafana
logger=settings t=2022-06-24T11:57:58.368048705+05:30 level=info msg="Path Logs" path=/var/log/grafana
logger=settings t=2022-06-24T11:57:58.368103463+05:30 level=info msg="Path Plugins" path=/var/lib/grafana/plugins
logger=settings t=2022-06-24T11:57:58.368161777+05:30 level=info msg="Path Provisioning" path=/etc/grafana/provisioning
logger=settings t=2022-06-24T11:57:58.368214425+05:30 level=info msg="App mode production"
  • What did you expect to happen?
    To strart properly

  • Can you copy/paste the configuration(s) that you are having problems with?

  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
    No errors in the log

  • Did you follow any online instructions? If so, what is the URL?

hi @jinujoseph

are you still blocked on this or did the issue resolve itself?

@mattabrams
This is still blocked. I just cant figure out where to start debugging since there are no error messages in the logfile.

One more input: I was shocked today to find that my Internet Data for the month was completely consumed. On investigation i found that the problem was with this grafana docker instance. It was downloading almost 200GB per day. I am not sure what it was doing but this was in an infinite restart loop and every time it restarted it would download some data. Again cant seem to figure out what it is downloading. For now i have shut it down.