Can't start Grafana 12.4.0: migration validation failed

Grafana Configuration
1

I am running grafana 12.4.0 on debian 13.3 for some home autmation dashboards. This has been working nicely for a long time, but for more the last day, I have not been able to access grafana. When I go to the grafana url, I just get firefox’ “Unable to connect” screen.

When looking into the logs, there are a lot of “Migration progress” messages, but at the end I get

logger=storage.unified.migrator t=2026-02-26T20:59:07.892371096+01:00 level=info msg=“start rebuilding index for resources” namespace=org-2 orgId=2 resources=“[folders.folder.grafana.app dashboards.dashboard.grafana.app]”
logger=storage.unified.migrator t=2026-02-26T20:59:12.89812991+01:00 level=info msg=“finished rebuilding index for resources” namespace=org-2 orgId=2 resources=“[folders.folder.grafana.app dashboards.dashboard.grafana.app]”
logger=storage.unified.migration_runner.folders-dashboards t=2026-02-26T20:59:12.898610174+01:00 level=info msg=“Count validation” resource=folders.folder.grafana.app namespace=org-2 legacy_count=1 unified_count=0 migration_summary_co
unt=0 rejected=0 history=0
logger=storage.unified.migration_runner.folders-dashboards t=2026-02-26T20:59:12.898673244+01:00 level=error msg=“Migration validation failed” org_id=2 error=“validator CountValidator failed: count mismatch for folders.folder.grafana.
app in namespace org-2: legacy has 1, unified has 0, rejected 0” duration=15.030621084s
logger=storage.unified.migration_runner.folders-dashboards t=2026-02-26T20:59:12.898687179+01:00 level=warn msg=“[WARN] Resource migration failed and is currently skipped.\nThis migration will be enforced in the next major Grafana rel
ease, where failures will block startup or resource loading.\n\nThis warning is intended to help you detect and report issues early.\nPlease investigate the failure and report it to the Grafana team so it can be addressed before the n
ext major release.” error=“migration validation failed for org 2 (org-2): validator CountValidator failed: count mismatch for folders.folder.grafana.app in namespace org-2: legacy has 1, unified has 0, rejected 0”
logger=unifiedstorage-migrator t=2026-02-26T20:59:12.898716512+01:00 level=error msg=“Executing migration failed” id=“folders and dashboards migration” error=“migration validation failed for org 2 (org-2): validator CountValidator fai
led: count mismatch for folders.folder.grafana.app in namespace org-2: legacy has 1, unified has 0, rejected 0” duration=2m40.544491524s
logger=unifiedstorage-migrator t=2026-02-26T20:59:12.898726965+01:00 level=error msg=“Exec failed” error=“migration validation failed for org 2 (org-2): validator CountValidator failed: count mismatch for folders.folder.grafana.app in
namespace org-2: legacy has 1, unified has 0, rejected 0” sql=“unified storage data migration: folders-dashboards”
logger=unifiedstorage-migrator t=2026-02-26T20:59:13.158520895+01:00 level=info msg=“Unlocking database”
logger=settings t=2026-02-26T20:59:13.594053243+01:00 level=info msg=“Starting Grafana” version=12.4.0 commit=d1729c53a7f44e2e58947eb44eb896c2fb1c30b3 branch=release-12.4.0 compiled=2026-02-26T20:59:13+01:00
logger=settings t=2026-02-26T20:59:13.594441789+01:00 level=info msg=“Unified migration configs enforced”

It seems to me as if grafana tries to migrate my setup,then the validation falls for some reason and it just retries the same migration that at the end fails again.

I have not touched any config files, I am using postgres as the main database, but grafana seems to be using sqlite for its own storage, everything is stored locally and there is free space on the partitions grafana is using. I did recently upgrade the server from debian 12, but I am fairly sure grafana has been working after the upgrade.

Any ways to get out of this loop and be able to use grafana again?

3

Is there any chance you have the feature toggle provisioningenabled? If so, try removing it.

4

Not as far as I can see, at the bottom of my grafana.ini, I have

[feature_toggles]
# enable features, separated by spaces
;enable =

at the top I have


# folder that contains provisioning config files that grafana will apply on startup and while running.
;provisioning = conf/provisioning

Under /etc/grafana/provisioning, there are a few directories with sample.yaml files in them, but the only thing not beeing commented out is some “apiVersion: 1”

5

So, to try to get things up again, without damaging the old setup, I started a new grafana in docker.

Then I had to query the grafana.db from the old setup to find the data sources

select * from data_source

and set up the same data sources in the new grafana, then I could do

select id,title from dashboard;

to get an overview of the dashboards

and then

.once dashboardtitle.json
select data from dashboard where title =’dashboardtitle’

I could then make a new dashboard with a panel having the datasources I wanted to use and open the json file for that panel. (settings - json model). In that json model, I could find the uids for the new data sources, like


"datasource": {
   "type": "grafana-postgresql-datasource",
   "uid": "afemhrq2hcxz4a"
}

I then had to copy out the uids, noting which belonged to which data source, then I deleted the model, opened dashboardtitle.json, copied out the entire text and pasted it into the json model in grafana, searched for each of the old data source uids and replaced with the right new one. Then I had the dashboard back, except for library panels.

Not too bad for my home setup with three dashboards I needed to rescue, it would have been a disaster if this happened at work with a lot more dashboards… (although there we do a test upgrade before we upgrade production, hopefully that would have caught that problem, but it would have left us with the problem how to upgrade to the new version)

1 Like
6

I have a similar problem with Ubuntu 22. With the last update to 12.4.0 i just read “service unavailable” with the browser. I had to restore a backup. I can’t understand if it’s OS related or just a problem with Grafana

7

I’m using docker and ended up downgrading from the latest tag to 12.3 and all is working again.

I tried recreating the container using 12.4 from old database backups, going back week and months, but always get the same error as in the first post.

1 Like
8

I have the same issue on Ubuntu 24.04 LTS. Had to rollback to version 12.3. It seems Grafana-Server service is not getting started. This is very unfortunate for what was a stable system of updates. I hope it gets fixed soon so users can take advantage of the new features.

9

Apologies for the trouble. A fix is now live in version 12.4.1. Please update and report back if you’re still seeing any issues. Thanks!