Hi,
We have a few deployment of grafana 5. In just one of our environments grafana is failing to start. The other environments all started without issue. The Mysql server it connects to has a single database called grafana. There are no tables in the database. There is just one Grafana instance connected to the database (normally there would be two but for testing we disabled one). The Grafana systemd service has a post service which runs to define a couple of datasources and change the admin password. Grafana fails to start with the following errors:
Jul 06 13:52:56 ip-172-24-34-148 grafana-post.sh[12588]: {"dbtype":"mysql","logger":"sqlstore","lvl":"info","msg":"Initializing DB","t":"2018-07-06T13:52:56.3130811Z"}
Jul 06 13:52:56 ip-172-24-34-148 grafana-post.sh[12588]: {"logger":"migrator","lvl":"info","msg":"Starting DB migration","t":"2018-07-06T13:52:56.318528063Z"}
Jul 06 13:52:56 ip-172-24-34-148 grafana-post.sh[12588]: {"id":"copy data account to org","logger":"migrator","lvl":"info","msg":"Executing migration","t":"2018-07-06T13:52:56.324045326Z"}
Jul 06 13:52:56 ip-172-24-34-148 grafana-post.sh[12588]: {"id":"copy data account to org","logger":"migrator","lvl":"info","msg":"Skipping migration condition not fulfilled","t":"2018-07-06T13:52:56.324483334Z"}
Jul 06 13:52:56 ip-172-24-34-148 grafana-post.sh[12588]: {"id":"copy data account_user to org_user","logger":"migrator","lvl":"info","msg":"Executing migration","t":"2018-07-06T13:52:56.325134411Z"}
Jul 06 13:52:56 ip-172-24-34-148 grafana-post.sh[12588]: {"id":"copy data account_user to org_user","logger":"migrator","lvl":"info","msg":"Skipping migration condition not fulfilled","t":"2018-07-06T13:52:56.32555145Z"}
Jul 06 13:52:56 ip-172-24-34-148 grafana-post.sh[12588]: {"id":"Add column created_by in dashboard - v2","logger":"migrator","lvl":"info","msg":"Executing migration","t":"2018-07-06T13:52:56.326185863Z"}
Jul 06 13:52:56 ip-172-24-34-148 grafana-post.sh[12588]: {"error":"Error 1060: Duplicate column name 'created_by'","id":"Add column created_by in dashboard - v2","logger":"migrator","lvl":"eror","msg":"Executing migration failed","t":"2018-07-06T13:52:56.326456497Z"}
Jul 06 13:52:56 ip-172-24-34-148 grafana-post.sh[12588]: {"error":"Error 1060: Duplicate column name 'created_by'","logger":"migrator","lvl":"eror","msg":"Exec failed","sql":"alter table `dashboard` ADD COLUMN `created_by` INT NULL ","t":"2018-07-06T13:52:56.326487374Z"}
Jul 06 13:52:56 ip-172-24-34-148 grafana-post.sh[12588]: {"error":"Sqlstore::Migration failed err: Error 1060: Duplicate column name 'created_by'\n","logger":"sqlstore","lvl":"eror","msg":"Fail to initialize orm engine","t":"2018-07-06T13:52:56.338063898Z"}
Tables are created in the grafana database
In the migration_log table there are errors at the end:
*************************** 56. row ***************************
id: 56
migration_id: Add column updated_by in dashboard - v2
sql: alter table `dashboard` ADD COLUMN `updated_by` INT NULL
success: 1
error:
timestamp: 2018-07-06 13:52:55
*************************** 57. row ***************************
id: 57
migration_id: Add column updated_by in dashboard - v2
sql: alter table `dashboard` ADD COLUMN `updated_by` INT NULL
success: 0
error: Error 1060: Duplicate column name 'updated_by'
timestamp: 2018-07-06 13:52:55
*************************** 58. row ***************************
id: 58
migration_id: Add column created_by in dashboard - v2
sql: alter table `dashboard` ADD COLUMN `created_by` INT NULL
success: 0
error: Error 1060: Duplicate column name 'created_by'
timestamp: 2018-07-06 13:52:55
*************************** 59. row ***************************
id: 59
migration_id: Add column created_by in dashboard - v2
sql: alter table `dashboard` ADD COLUMN `created_by` INT NULL
success: 0
error: Error 1060: Duplicate column name 'created_by'
timestamp: 2018-07-06 13:52:56
59 rows in set (0.00 sec)
If we delete the column we get a different but similar error, this time referring to a duplicate index.
We tried playing whack-a-mole for a bit but ran out of energy.
Any suggestions to fix this would be most welcome.
Regards,
Matthew