Grafana Dashboard Provisioning error

Hi,

I am using Grafana 5.1.2. While installing it using the Docker image, I am getting below error:

t=2018-07-11T11:12:52+0000 lvl=eror msg=“Executing migration failed” logger=migrator id=“create dashboard_provisioning” error=“disk I/O error”
t=2018-07-11T11:12:52+0000 lvl=eror msg=“Exec failed” logger=migrator error=“disk I/O error” sql=“CREATE TABLE IF NOT EXISTS dashboard_provisioning (\nid INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL\n, dashboard_id INTEGER NULL\n, name TEXT NOT NULL\n, external_id TEXT NOT NULL\n, updated DATETIME NOT NULL\n);”
t=2018-07-11T11:12:52+0000 lvl=eror msg=“Fail to initialize orm engine” logger=sqlstore error=“Sqlstore::Migration failed err: disk I/O error\n”

My Dockerfile looks like:

FROM —***—/baseimages/rhel:7

ARG GRAFANA_URL=“https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana-5.1.2.linux-x64.tar.gz
ARG GF_UID=“472”
ARG GF_GID=“472”

ENV PATH=/usr/share/grafana/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
GF_PATHS_CONFIG="/etc/grafana/grafana.ini"
GF_PATHS_DATA="/var/lib/grafana"
GF_PATHS_HOME="/usr/share/grafana"
GF_PATHS_LOGS="/var/log/grafana"
GF_PATHS_PLUGINS="/var/lib/grafana/plugins"
GF_PATHS_PROVISIONING="/etc/grafana/provisioning"

RUN yum -y update && yum install -qq -y tar libfontconfig curl ca-certificates &&
mkdir -p “$GF_PATHS_HOME/.aws” &&
curl “$GRAFANA_URL” | tar xfvz - --strip-components=1 -C “$GF_PATHS_HOME” &&
yum autoremove -y && yum clean all &&
groupadd -r -g $GF_GID grafana &&
useradd -r -u $GF_UID -g grafana grafana &&
mkdir -p “$GF_PATHS_PROVISIONING/datasources”
“$GF_PATHS_PROVISIONING/dashboards”
“$GF_PATHS_LOGS”
“$GF_PATHS_PLUGINS”
“$GF_PATHS_DATA” &&
cp “$GF_PATHS_HOME/conf/sample.ini” “$GF_PATHS_CONFIG” &&
cp “$GF_PATHS_HOME/conf/ldap.toml” /etc/grafana/ldap.toml &&
chown -R root:root /etc/grafana &&
chmod -R a+r /etc/grafana &&
chown -R grafana:grafana “$GF_PATHS_DATA” “$GF_PATHS_HOME/.aws” “$GF_PATHS_LOGS” “$GF_PATHS_PLUGINS” &&
chmod 777 “$GF_PATHS_DATA” “$GF_PATHS_HOME/.aws” “$GF_PATHS_LOGS” “$GF_PATHS_PLUGINS”

EXPOSE 3000

COPY ./docker/gf/bin/run.sh /run.sh

USER grafana
WORKDIR /
ENTRYPOINT [ “/run.sh” ]

Grafana Logs:

installing camptocamp-prometheus-alertmanager-datasource @ 0.0.6
from url: https://grafana.com/api/plugins/camptocamp-prometheus-alertmanager-datasource/versions/0.0.6/download
into: /var/lib/grafana/plugins

â Installed camptocamp-prometheus-alertmanager-datasource successfully

Restart grafana after installing plugins .

t=2018-07-11T11:12:50+0000 lvl=info msg=“Starting Grafana” logger=server version=5.1.2 commit=c3c690e21 compiled=2018-05-09T09:00:18+0000
t=2018-07-11T11:12:50+0000 lvl=info msg=“Config loaded from” logger=settings file=/usr/share/grafana/conf/defaults.ini
t=2018-07-11T11:12:50+0000 lvl=info msg=“Config loaded from” logger=settings file=/etc/grafana/grafana.ini
t=2018-07-11T11:12:50+0000 lvl=info msg=“Config overridden from command line” logger=settings arg=“default.paths.data=/var/lib/grafana”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Config overridden from command line” logger=settings arg=“default.paths.logs=/var/log/grafana”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Config overridden from command line” logger=settings arg=“default.paths.plugins=/var/lib/grafana/plugins”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Config overridden from command line” logger=settings arg=“default.log.mode=console”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Config overridden from Environment variable” logger=settings var=“GF_PATHS_DATA=/var/lib/grafana”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Config overridden from Environment variable” logger=settings var=“GF_PATHS_LOGS=/var/log/grafana”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Config overridden from Environment variable” logger=settings var=“GF_PATHS_PLUGINS=/var/lib/grafana/plugins”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Config overridden from Environment variable” logger=settings var=“GF_PATHS_PROVISIONING=/etc/grafana/provisioning”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Config overridden from Environment variable” logger=settings var=“GF_SECURITY_ADMIN_USER=admin”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Config overridden from Environment variable” logger=settings var=“GF_SECURITY_ADMIN_PASSWORD=*********”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Path Home” logger=settings path=/usr/share/grafana
t=2018-07-11T11:12:50+0000 lvl=info msg=“Path Data” logger=settings path=/var/lib/grafana
t=2018-07-11T11:12:50+0000 lvl=info msg=“Path Logs” logger=settings path=/var/log/grafana
t=2018-07-11T11:12:50+0000 lvl=info msg=“Path Plugins” logger=settings path=/var/lib/grafana/plugins
t=2018-07-11T11:12:50+0000 lvl=info msg=“Path Provisioning” logger=settings path=/etc/grafana/provisioning
t=2018-07-11T11:12:50+0000 lvl=info msg=“App mode production” logger=settings
t=2018-07-11T11:12:50+0000 lvl=info msg=“Initializing DB” logger=sqlstore dbtype=sqlite3
t=2018-07-11T11:12:50+0000 lvl=info msg=“Starting DB migration” logger=migrator
t=2018-07-11T11:12:50+0000 lvl=info msg=“Executing migration” logger=migrator id=“copy data account to org”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Skipping migration condition not fulfilled” logger=migrator id=“copy data account to org”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Executing migration” logger=migrator id=“copy data account_user to org_user”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Skipping migration condition not fulfilled” logger=migrator id=“copy data account_user to org_user”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Executing migration” logger=migrator id=“Drop old table account”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Executing migration” logger=migrator id=“Drop old table account_user”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Executing migration” logger=migrator id=“Update org table charset”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Executing migration” logger=migrator id=“Update org_user table charset”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Executing migration” logger=migrator id=“Migrate all Read Only Viewers to Viewers”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Executing migration” logger=migrator id=“create dashboard table”
t=2018-07-11T11:12:50+0000 lvl=info msg=“Executing migration” logger=migrator id=“add index dashboard.account_id”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“add unique index dashboard_account_id_slug”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“create dashboard_tag table”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“add unique index dashboard_tag.dasboard_id_term”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“drop index UQE_dashboard_tag_dashboard_id_term - v1”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Rename table dashboard to dashboard_v1 - v1”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“create dashboard v2”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“create index IDX_dashboard_org_id - v2”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“create index UQE_dashboard_org_id_slug - v2”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“copy dashboard v1 to v2”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“drop table dashboard_v1”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“alter dashboard.data to mediumtext v1”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Add column updated_by in dashboard - v2”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Add column created_by in dashboard - v2”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Add column gnetId in dashboard”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Add index for gnetId in dashboard”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Add column plugin_id in dashboard”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Add index for plugin_id in dashboard”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Add index for dashboard_id in dashboard_tag”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Update dashboard table charset”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Update dashboard_tag table charset”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Add column folder_id in dashboard”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Add column isFolder in dashboard”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Add column has_acl in dashboard”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Add column uid in dashboard”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Update uid column values in dashboard”
t=2018-07-11T11:12:51+0000 lvl=info msg=“Executing migration” logger=migrator id=“Add unique index dashboard_org_id_uid”
t=2018-07-11T11:12:52+0000 lvl=info msg=“Executing migration” logger=migrator id=“Remove unique index org_id_slug”
t=2018-07-11T11:12:52+0000 lvl=info msg=“Executing migration” logger=migrator id=“Update dashboard title length”
t=2018-07-11T11:12:52+0000 lvl=info msg=“Executing migration” logger=migrator id=“Add unique index for dashboard_org_id_title_folder_id”
t=2018-07-11T11:12:52+0000 lvl=info msg=“Executing migration” logger=migrator id=“create dashboard_provisioning”
t=2018-07-11T11:12:52+0000 lvl=eror msg=“Executing migration failed” logger=migrator id=“create dashboard_provisioning” error=“disk I/O error”
t=2018-07-11T11:12:52+0000 lvl=eror msg=“Exec failed” logger=migrator error=“disk I/O error” sql=“CREATE TABLE IF NOT EXISTS dashboard_provisioning (\nid INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL\n, dashboard_id INTEGER NULL\n, name TEXT NOT NULL\n, external_id TEXT NOT NULL\n, updated DATETIME NOT NULL\n);”
t=2018-07-11T11:12:52+0000 lvl=eror msg=“Fail to initialize orm engine” logger=sqlstore error=“Sqlstore::Migration failed err: disk I/O error\n”

Could you help me resolve it.

The issue occurs when persistence is enabled. I have added below lines to deployment.yaml, still the issue is not resolved:

  securityContext:
    runAsUser: 472
    fsGroup: 472

“/var/lib/grafana” directory should have the ownership of grafana:grafana as per the Dockerfile instructions, but its having the ownership of nobody:nobody verified at deployment.

Is this a known issue?

Regards,
Rajya